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HUMAN  FACTORS  CHARACTERISTICS  OF  THE  JOINT  TACTICAL  FUSION  TEST  BED: 
FIELD  TEST  467  RESULTS 


Foreword 


The  Fort  Hood  Field  Unit  of  the  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  conducts  research  in  a  variety  of  areas  related  to  the  needs  of  the 
Army  in  the  field. 

The  US  Army  is  in  the  process  of  obtaining  a  wide  variety  of  automated  equipments 
in  an  effort  to  overcome  its  disadvantage  in  numbers  compared  to  potential  adversaries. 
As  a  part  of  this  effort,  the  TRADOC  Combined  Arms  Test  Activity  (TCATA)  evaluated 
the  Joint  Tactical  Fusion  Test  Bed-Army  (JTFBT-A).  The  primary  objective  of  the 
evaluation  was  to  determine  the  capability  of  automated  systems  to  perform  selected 
functions  anticipated  for  inclusion  in  automated  intelligence  analysis  systems.  To  be  of 
use,  these  future  systems  must  aid  the  analyst  in  performing  these  functions  both  more 
rapidly  and  more  accurately  than  with  the  currently  employed  manual  methods. 

This  report  describes  the  human  factors  evaluation  of  the  JTFTB-A  conducted  as  a 
part  of  the  TCATA  test.  Questionnaires,  interviews,  direct  observation  and  physical 
measures  of  the  equipment  and  environment  were  employed.  Where  appropriate,  the 
potential  effects  of  the  human  factors  characteristics  on  overall  system  performance  are 
discussed.  The  results  of  this  research  have  been  incorporated  into  TCATA  Test  Report 
FT  467,  Joint  Tactical  Fusion  Test  Bed- Army  (JTFTB-A)  Final  Report,  May  1982. 

The  research  described  in  this  report  was  performed  by  the  Human  Resources 
Research  Organization  (HumRRO)  under  Contract  No.  MDA903-79-C-0191.  This 
research  is  responsive  to  the  special  requirements  and  objectives  of  RDTE  Project 
2Q263743A794,  "Human  Factors  and  Training  Research  in  Military  Organizations  and 
Systems,"  FY  1981  Work  Program. 


HUMAN  FACTORS  CHARACTERISTICS  OF  THE  JOINT  TACTICAL  FUSION  TEST  BED: 
FIELD  TEST  467  RESULTS 

Executive  Summary 


Requirement: 

Automated  systems  increasingly  are  seen  as  a  way  for  the  Army  to  make  up  for  its 
disadvantage  in-numbers  compared  with  potential  adversaries.  An  automated  All  Source 
Analysis  System  (AS AS)  for  intelligence  data  is  one  such  system  being  considered  by  the 
Army.  Because  intelligence  analysis  is  highly  complex  and  largely  subjective  in  nature 
with  few  well-defined  procedures,  the  Army  chose  to  evaluate  the  capability  of 
automated  systems  to  perform  certain  ASAS-related  functions  by  means  of  a  test  bed. 
The  Joint  Tactical  Fusion  Test  Bed-Army  (JTFTBA)  was  the  system  employed  for  the 
evaluation.  One  of  the  objectives  of  the  evaluation  was  to  collect  data  on  human  factors 
aspects  of  the  system.  Although  the  stated  objective  only  called  for  the  ’’collection”  of 
human  factors  data,  an  attempt  is  made  in  this  report  to  relate  these  data  to  the 
presumed  ASAS-related  functions. 

Procedures: 

Interviews,  questionnaires,  direct  observation  and  physical  measurements  were 
employed  to  obtain  the  human  factors  data.  Each  operator  received  a  structured 
interview  following  training  on  the  operator  terminal.  Items  requested  information  about 
specific  functions  of  the  terminal  and  general  opinions  about  the  system  as  a  whole. 
Human  factors  questionnaires  were  administered  at  the  completion  or  training  and  again 
during  the  Command  Post  Exercise  (CPX).  These  questionnaires  covered  safety,  the 
adequacy  of  features  and  characteristics  of  the  system,  and  physical  complaints.  Human 
factors  specialists  observed  operators  throughout  the  evaluation,  and  when  possible  and 
appropriate,  conducted  informal  interviews  to  obtain  a  better  understanding  of  any 
problems  that  occurred.  Evironmental  measures  were  obtained  in  three  locations,  and 
physical  measurements  of  the  operator's  station  were  obtained  to  determine  conformity  to 
published  human  factors  standards. 

Findings: 

In  general,  operators  felt  that  their  performance  with  the  system  would  be  superior 
to  their  performance  employing  the  manual  methods  currently  used  by  intelligence  units. 
They  liked  the  speed  with  which  some  functions  were  performed  and  felt  that  the  ability 
to  perform  a  number  of  functions  without  the  need  for  interaction  with  other  analysts  was 
highly  desirable.  Overall,  the  78  individual  functions  performed  using  the  keyboard  were 
highly  rated.  However,  oprators  strongly  disliked  the  cursor  controls  (the  cursor  could 
easily  be  lost)  and  felt  that  the  AGGREATION,  PURGE  and  TARGET  DESIGNATION 
functions  were  too  time  consuming  and  tedious.  They  also  felt  that  redraw  times  for  the 
map  (often  30  seconds)  were  excessive.  The  two  other  major  complaints  were  that  error 
messages  were  difficult  to  interpret,  and  that  alertng  for  incoming  messages  was 
inadequate. 

The  terminal  configuration  failed  to  conform  to  published  standards  in  a  number  of 
areas.  Most  noteably,  controls  for  several  critical  and  frequently  used  functions  were 


located  in  areas  which  were  difficult  to  reach.  Also,  both  tall  and  short  operators  had 
difficulty  in  "fitting"  themselves  to  the  terminal.  Finally,  lefthanded  operators  felt  the 
terminal  was  difficult  to  use  as  the  writing  area  was  on  the  right  side  of  the  terminal. 

Noise  levels  in  the  vans  were  above  recommended  maxima  for  areas  where  voice  and 
telephonic  communications  are  required.  Temperatures  were  below  recommended  stan¬ 
dards  but  relative  humidity  was  within  tolerance.  However,  since  the  system  was  not 
configured  for  actual  deployment,  these  measures  can  only  serve  as  guides  to  designers. 

Utilization  of  Findings: 

The  findings  of  this  human  factors  research  have  been  incorporated  into  TCATA 
Test  Report  FT  467,  Joint  Tactical  Fusion  Test  Bed- Army  (JTFTB-A)  Final  Report,  May 
1982,  and  will  be  used  to  design  future  automated  systems,  especially  those  performing 
ASAS-related  functions. 
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HUMAN  FACTORS  CHARACTERISTICS  OF  THE  JOINT  TACTICAL 
FUSION  TEST  BED:  FIELD  TEST  467  RESULTS 


Introduction 

Automated  systems  increasingly  are  seen  as  a  way  for  the  Army  to  make  up  for  its 
disadvantages  in  numbers  compared  with  potential  adversaries.  The  early  use  of  these 
automated  systems  occurred  in  the  fields  of  weapons  control  and  administration,  but  much 
of  the  recent  development  has  been  in  the  fields  of  Command,  Control,  Communications 
and  Intelligence  (C30.1  Such  systems  are  intended  to  provide  the  commander  with  better 
ways  to  manage  the  tremendous  amount  of  information  available  on  the  modern 
battlefield.  Whereas  the  earlier  systems  were  designed  to  solve  well  understood  problems 
such  as  how  to  lay  a  gun  on  target,  the  C3i  systems  must  deal  with  poorly  understood 
problems  where  there  are  no  well  defined  procedures  or  solutions. 

One  such  system,  the  Joint  Tactical  Fusion  Test  Bed-Army  (JTFTB-A),  was 
evaluated  by  the  TRADOC  Combined  Arms  Test  Activity  (TCATA)  from  June  through 
November  of  1981.2  The  evaluation  had  four  objectives.  The  primary  objective  was  "to 
assess  the  capability  of  the  JTFTB-A  to  perform  selected  ASAS-related  (All  Source 
Analysis  System  for  intelligence)  functions."*  Another  objective,  and  the  subject  of  this 
report,  was  to  collect  data  on  human  factors  aspects  of  the  system.  The  remaining 
objectives  were  to  collect  data  on  operator  training  and  to  collect  data  on  reliability, 
availability,  and  maintainability  of  the  system.  Although  the  stated  objective  only  called 
for  the  "collection"  of  human  factors  data,  an  attempt  is  made  in  this  report  to  relate 
these  data  to  overall  system  functioning,  especially  to  presumed  ASAS-related  functions. 
By  doing  so,  it  is  hoped  that  this  report  will  meet  its  primary  objective  —  that  of 
providing  guidance  to  designers  of  future  C^I  systems. 

The  JTFTB-A  represented  a  change  in  direction  by  the  Army  in  its  approach  to  C^I 
systems.  Earlier  attempts  had  either  tried  to  automate  a  large  part  of  the  command  and 
control  process  in  one  system  or  had  concentrated  on  a  very  small  aspect  of  it  such  as  fire 
control.*  The  Tactical  Operations  System  (TOS)  attempted  to  automate  the  commander's 
full  range  of  C^i  functions.  So  many  different  functions  were  incorporated  into  this 


^Institute  for  Defense  Analyses.  Computers  in  command  and  control  (TR  61-12). 
Washington,  DC:  Research  and  Engineering  Support  Division,  November  1961.  (DTIC  No. 
AD  251  997) 

2US  Army  Training  and  Doctrine  Command  Combined  Arms  Test  Activity.  TCATA 
Test  Report  FT  467:  Joint  Tactical  Fusion  Test  Bed-Army  (JTFTB-A)  (RCS  ATCD-8). 
Fort  Hood,  TX:  Author,  May  1982. 

3Ibid. 

*US  Army  Combined  Arms  Combat  Development  Activity.  Architectural  concept 
for  1985  for  U.S.  Army  Tactical  Command,  Control,  Communication  and  Intelligence  (cjl) 


system  that  it  did  none  of  them  well  and  the  development  efforts  had  to  be  discontinued.® 
By  contrast,  the  Tactical  Fire  Direction  system  (TACFIRE),  which  was  originally  planned 
as  an  element  of  TOS,  concentrated  on  the  limited  problem  of  using  an  automated  system 
to  direct  the  division's  artillery.®  This  system  is  now  fielded  and  being  used  successfully. 
Its  usefulness  is  limited,  however,  because  it  does  not  have  automatic  information 
exchange  either  with  the  computer  equipment  at  the  battery  level  or  with  any  high  level 
systems. 

The  JTFTB-A  represents  an  attempt  to  choose  some  of  the  best  of  both 
approaches.7  Like  TACFIRE,  it  focuses  on  a  small  part  of  the  command  and  control 
problem,  the  fusion  of  intelligence  data.  According  to  hearings  in  the  US  House  of 
Representatives: 

A  tactical  fusion  center  is  a  means  of  accepting  intelligence 
and  command  and  control  data  from  many  different  sources  and 
"fusing"  them  in  a  coherent,  readable  and  understandable 
manner  [sic]  to  display  to  a  military  commander  what  is 
happening  on  the  battlefield  so  that  the  commander  may  decide 
what  tactical  options  are  available  to  him.® 

By  concentrating  on  the  limited  problem  of  fusing  intelligence  reports  from  various 
sources,  both  electronic  and  human,  the  JTFTB-A  could  be  designed  around  procedures 
that  had  already  been  used  in  other  automated  systems  and  could  be  designed  into  a 
hardware  package  small  enough  to  transport  into  the  field. 

The  JTFTB-A  was  not  designed  with  the  notion  that  such  a  system  would  totally 
supplant  the  current  manual  intelligence  operation  which  supports  the  corps  commander. 
Rather,  it  was  designed  to  handle  many  of  the  routine,  time-consuming  functions  that 
take  much  of  the  intelligence  analyst's  efforts.®  This  leaves  the  analyst  free  to  handle 
the  higher-level  cognitive  tasks  of  deciding  what  the  enemy's  order  of  battle  is  and  what 


®US  General  Accounting  Office.  Tactical  operations  system  development  should  not 
continue  as  planned  (LCD-80- 17).  Washington,  DC:  Author,  November  20,  1979. 

® A.  M.  Haynes,  P.  L.  LaPointe,  H.  L.,  Cooke,  Sr  J.  A.  Underwood,  in.  Tactical  fire 
direction  (TACFIRE)  operational  test  III  (TCATA  Test  Report  No.  OT  056).  Fort  Hood, 
TX:  TRADOC  Combined  Arms  Test  Activity  (TCATA),  August  1978. 

7P.  T.  Marston,  A.  L.  Kubala,  Sr  R.  G.  Cooper.  Human  factors  considerations  in  the 
battlefield  exploitation  and  target  acquisition  (BETA)  system:  A  preliminary  evaluation 
(FR-MTRD(TX)-81-12).  Alexandria,  VA:  Human  Resources  Research  Organization, 
February  1981. 

8 US  House  of  Representatives.  Department  of  Defense  Appropriations  Bill,  1981 
(HR  96-1317).  Washington,  DC:  US  Government  Printing  Office,  September  11,  1980. 

A.  Montgomery,  J.  R.  Thompson,  Sr  R.  V.  Katter.  Human  processes  in 
intelligence  analysis:  Phase  I  overview  (Research  Report  1237).  Alexandria,  VA:  US 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  1979. 
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the  enemv  commander  intends  to  dc.10  Thus,  the  JTFTB-A  functions  as  a  decision  aiding 
system.  II  It  allows  the  analyst  to  access  the  information  it  has  stored  in  its  data  base 
about  what  the  intelligence  sources  have  detected.  It  presents  this  information  to  the 
analyst  either  graphically  as  a  location  on  a  map  of  the  battlefield  or  as  a  report  which 
can  be  displayed  or  printed.  By  providing  a  common  data  base,  it  allows  several  analysts 
to  work  on  the  same  information  and  share  their  results  very  quickly.  All  of  these 
features  should  allow  the  analyst  to  manipulate  the  information  into  a  form  conducive  to 
evaluating  the  enemy's  intentions  and  assets  much  faster  and  more  accurately  than  with 
current  manual  systems. 

All  operator-induced  data  manipulations  take  place  through  the  use  of  the  inter¬ 
active  terminal.  Because  this  interface  is  so  important  to  successful  operation,  the  bulk 
of  the  human  factors  data  collection  focused  on  this  device.  Since  the  system  was  a  test 
bed,  no  operational  concept  had  been  developed.  Hence,  the  research  effort  concentrated 
on  those  characteristics  of  the  system  that  are  likely  to  be  a  part  of  any  future 
intelligence  analysis  system  rather  than  those  specific  to  JTFTB-A.  For  example,  the 
configuration  of  the  terminal  keyboard  is  probably  specific  to  this  system,  while  the 
symbolic  presentation  of  forces  on  a  map-like  situation  display  is  a  likely  feature  of  any 
intelligence  analysis  system.  Nevertheless,  the  terminal  design  was  evaluated  as  it  was 
felt  that  the  information  gleaned  could  be  of  use  to  designers  of  future  systems. 

The  most  important  feature  of  the  user-computer  interface  is  the  interactive 
dialogue.  12, 13  This  dialogue  consists  of  all  the  interactions  that  the  operator  has  with  the 
system  to  make  it  accomplish  the  required  tasks.  From  the  terminal,  the  elements  are 
prompts,  menus,  forms  to  fill  out,  error  messages,  reports  and  map  displays.  From  the 
operators,  the  elements  are  commands,  menu  selections,  information  for  forms,  position¬ 
ing  of  the  cursor,  and  free- text  messages.  If  the  dialogue  characteristics  facilitate 
communications  between  the  user  and  the  computer  system,  then  the  user  should  be  able 
to  obtain  exactly  what  is  wanted  with  a  minimum  of  effort.  To  the  extent  that  the 
dialogue  does  not  facilitate  the  operator's  job,  the  system  may  prove  difficult  to  learn, 
difficult  to  use,  and  a  source  of  physical  and  mental  problems.  Hence,  an  evaluation  of 
the  adequacy  of  the  dialogue  is  essential  and  is  given  consideration  in  this  report. 


1°R.  G.  Cooper,  P.  T.,  Marston,  fir  A.  L.  Kubala.  Human  factors  in  automated  cj| 
systems  (Draft).  Alexandria,  VA;  Human  Resources  Research  Organization,  March  1981. 

Hr.  C.  Goldstein.  The  substantive  use  of  computers  for  intelligence  activities 
(MAC  TM  21).  Cambridge,  MS:  Massachusetts  Institute  of  Technology,  April  1971.  (DTIC 
NO.  AD  721  618) 

12j.  Martin.  Design  of  man-computer  dialogues.  Englewood  Cliffs,  NJ: 
Prentice-Hall,  1973. 

13h.  R.  Ramsey  If  M.  Atwood.  Human  factors  in  computer  systems:  A  review  of 
literature  (Technical  Report  SAI-79-1 11-DEN).  Englewood,  CO:  Science  Applications, 
Inc.,  September  1979.  (DTIC  No.  AD  A075  679) 


A  recent  concern  with  systems  that  have  a  Visual  Display  Terminal  (VDT)  using  a 
cathode  ray  tube  to  present  the  information  is  the  possibility  of  health  hazards  to  the 
operators  when  operating  the  terminal  many  hours  per  day.  Recent  studies  have  reported 
that  many  VDT  operators  complained  of  various  physical  and  emotional  symptoms.  14,15 
The  JTFTB-A  operator's  position  is  likely  to  require  long  hours  under  high  stress  at  the 
display,  so  health  problems  are  of  concern  in  the  system  design  and  were  considered  in  the 
human  factors  evaluation. 

In  order  to  put  the  results  of  this  human  factors  data  collection  effort  in 
perspective,  it  is  necessary  to  understand  how  the  JTFTB-A  system  is  designed  to  work. 
As  mentioned  earlier,  no  formal  concept  of  operations  for  the  system  had  been  developed. 
However,  as  the  authors  worked  with  the  system,  they  were  able  to  piece  together  a 
reasonable  concept  of  operation  from  interviews  with  test  and  project  office  personnel 
and  from  the  various  technical  documents  provided  by  the  contractors.  A  description  of 
the  system  based  primarily  on  those  documents^,  17,18  js  presented  in  Appendix  A  and 
should  be  reviewed  by  readers  of  this  report  who  are  not  familiar  with  the  JTFTB-A 
system. 


Methods,  Analysis  and  Participants 


Data  Collection  Methods 


The  human  factors  data  for  the  JTFTB-A  were  collected  using  a  combination  of 
interviews,  questionnaires  and  direct  observation.  The  researchers  observed  the  test  bed 
operation  and  operator  training  from  the  time  the  system  first  arrived  at  Fort  Hood, 
Texas,  through  the  Command  Post  Exercise  (CPX)  which  concluded  the  first  year's  evalu¬ 
ation.  Informal  interviews  were  conducted  throughout  the  test  with  all  of  the  test  team 
members  in  order  to  learn  more  about  the  system.  Additionally,  all  of  the  operators  were 
formally  interviewed  during  or  shortly  after  their  training,  and  questionnaires  were  given 
to  them  after  their  training  and  during  the  CPX.  Each  phase  of  the  test  was  observed  to 


14M.  J.  Smith,  B.  G.  F.  Cohen,  If  L.  W.  Stammerjohn,  Jr.  An  investigation  of  health 
complaints  and  job  stress  in  video  display  operations.  Human  Factors  1981,  23,  387-400. 

15L.  W.  Stammerjohn,  Jr.,  M.  J.  Smith,  Sr  B.  G.  F.  Cohen.  Evaluation  of  work 
station  design  factors  in  VDT  operations.  Human  Factors,  1981,  23,  401-412. 

16trw  Defense  and  Space  Systems  Group.  BETA  operator  positional  handbook 
(CDRL  A009,  FSCM  No.  11982).  Redondo  Beach,  CA:  Author,  April  1,  1980. 

1?TRW  Defense  and  Space  Systems  Group.  BETA  architecture  and  process  flows 
(SS22-47B,  FSCM  No.  11982).  Redondo  Beach,  CA:  Author,  September  28,  1979.  “““ 

18trw  Defense  and  Space  Systems  Group.  System  specifications  for  BETA  test  bed 
(SY16-12-C,  FSCM  No.  11982).  Redondo  Beach,  CA:  Author,  January  26,  1979. 
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see  how  the  operators  interacted  with  the  test  bed.  Finally,  the  operator  terminal  was 
measured  and  photographed  to  determine  how  well  it  conformed  to  existing  human  factors 
engineering  standards. 

Operator  interviews.  Each  operator  received  a  structured  interview  (see  Appen¬ 
dix  B)  aneFTecelvTngTraining  on  the  JTFTB-A  terminal.  It  consisted  of  101  items 
requesting  information  about  specific  functions  of  the  operator  terminal  and  general 
opinions  about  the  way  the  system  functioned  as  a  whole.  The  first  43  questions  dealt 
with  function  keys  that  controlled  a  single  action,  such  as  moving  the  cursor  or  deleting  a 
character.  The  next  35  questions  dealt  with  the  more  complex  functions  that  typically 
started  an  interactive  dialogue  which  tailored  a  function  to  a  specific  requirement. 
Examples  of  complex  functions  are  CREATE  QUERY  and  INPUT  CONTROL  MEASURES. 
For  both  simple  and  complex  functions  the  operator  was  asked  (a)  "Have  you  used  this  key 
[function!?;"  (b)  "How  much  did  you  use  it?;"  (c)  "How  well  did  it  work?;"  (d)  "What 
problems  did  you  have  with  it?;"  and  (e)  "How  would  you  change  it?"  In  some  cases  it  was 
more  logical  to  ask  about  a  group  of  similar  keys,  such  as  the  cursor  positioning  functions, 
so  the  questions  were  phrased  in  the  plural.  In  most  cases  the  answers  to  items  (a),  (b), 
and  (c)  involved  only  one  or  two  words  while  answers  to  items  (d)  and  (e)  tended  to  be 
more  elaborate.  The  last  23  questions  of  the  interview  dealt  with  the  operator's  opinions 
concerning  the  use  of  the  system  to  carry  out  specific  tasks,  how  the  system  might  be 
changed,  and  overall  impressions. 

The  interview  was  given  either  in  a  single  session  or  over  two  days,  depending  on  the 
operator's  schedule  and  lasted  from  30  minutes  to  two  hours,  depending  on  how  much  the 
individual  had  to  say. 

The  first  group  of  operators,  trained  in  the  summer,  were  in  the  process  of  using  the 
system  for  the  single  function  tests  when  they  were  interviewed,  so  they  were  asked 
separately  about  each  of  the  78  functions.  The  second  group  of  operators,  trained  in  the 
fall,  did  not  have  as  much  experience  using  the  terminal,  so  after  the  first  two  operators 
were  interviewed,  an  abbreviated  procedure  was  developed.  The  operator  was  shown  a 
diagram  of  the  keyboard  and  asked  to  point  out  which  functions  he  or  she  had  used.  As 
each  function  was  indicated,  the  last  four  of  the  five  questions  listed  above  were  asked 
for  that  function.  After  completing  this  portion,  the  operator  received  the  same  final  23 
items  as  the  first  group.  Although  it  was  clear  that  the  second  group  of  operators  did  not 
know  as  much  about  JTFTB-A  as  the  first  group,  this  procedure  seemed  to  produce  about 
the  same  information  while  shortening  the  interviews  considerably. 

Human  factors  questionnaire.  The  operators  were  given  a  human  factors  question¬ 
naire  (Appendix  C)  the  last  day  of  each  two-week  training  period  and  were  asked  to  rate 
the  quality  of  various  system  features.  The  areas  covered  by  the  questionnaire  were  con¬ 
trols,  workspace,  environment,  graphics  display,  alphanumeric  display,  and  safety.  The 
items  for  all  of  the  areas  except  safety  were  rated  on  a  five-point  scale  running  from  1  to 
5.  These  values  were  labeled  "very  inadequate,"  "inadequate,"  "borderline,"  "adequate," 
and  "very  adequate,"  respectively.  The  items  in  the  safety  area  referred  to  specific  parts 
of  the  system  which  were  rated  as  having  or  not  having  electrical,  thermal,  structural, 
mechanical,  visual,  or  auditory  safety  hazards.  It  should  be  noted  that  the  information  on 
environment  and  safety  were  of  limited  value  because  the  operator  terminals  were 
installed  in  a  tunnel  rather  than  in  mobile  shelters  as  they  likely  would  be  in  a  fielded 
system. 


CPX  questionnaire.  The  CPX  questionnaire  (Appendix  D)  was  given  to  the  operators 
on  duty  during  each  shih  of  the  exercise.  The  data  collection  for  this  phase  was  designed 
not  to  interfere  with  the  ongoing  CPX  operations.  As  a  result,  the  questionnaire  was  not 
given  at  the  same  time  during  each  shift  and  operators  were  occasionally  missed.  The 

?uestionnaire  consisted  of  two  parts:  a  rating  scale  and  a  list  of  health  complaints.  The 
irst  part  had  eight  items  which  described  major  characteristics  of  the  system  and  the 
CPX  operations.  These  were  rated  on  the  same  five-point  scale  used  with  the  human 
factors  questionnaire.  The  second  part  of  the  questionnaire  asked  the  operators  to  state 
whether  they  experienced  any  of  11  common  physical  complaints.  The  complaint  data 
were  obtained  at  the  beginning  and  ending  of  each  shift,  while  the  ratings  were  taken  near 
the  middle. 

Direct  observation.  The  human  factors  researchers  were  present  for  all  phases  of 
the  test.  They  observed  the  operators  learning  about  and  using  the  various  functions. 
These  observations  were  supplemented  with  direct  questions  about  what  an  operator  was 
doing  when  this  would  not  interfere  with  other  parts  of  the  test.  Notes  were  taken  both 
on  what  the  operators  did  well  and  what  problems  they  had.  Note  was  taken  of  how  they 
responded  to  equipment  and  software  problems.  When  a  situation  arose  where  it  was  not 
clear  what  was  going  on  at  an  operator  terminal,  both  the  test  personnel  and  the  support 
contractor  personnel  were  questioned  to  determine  what  might  be  happening.  This  proved 
to  be  a  useful  procedure  since  many  of  the  operators'  problems  with  functions  resulted 
from  "bugs"  in  the  system.  It  was  difficult  for  the  operators  to  know  what  to  do  when 
functions  did  not  work  as  described  in  the  training.  After  the  observers  had  been  with  the 
system  for  some  time,  the  operators  and  test  personnel  began  volunteering  human  factors 
related  information.  This  was  particularly  useful  since  there  were  never  enough  test 
personnel  to  cover  everything  that  was  happening. 

Physical  measurements.  Many  human  factors  engineering  standards  relate  to  the 
size  and  shape  of  equipment.  The  dimensions  of  the  operator  terminal  were  obtained  to 
the  nearest  millimeter  using  a  steel  tape  measure.  These  were  compared  with  engineering 
drawings  held  by  the  support  contractors  to  verify  their  accuracy.  Discrepancies  were 
resolved  by  making  the  measurement  again.  These  measurements  were  employed  to 
calculate  the  critical  dimensions  for  this  type  of  equipment  referenced  in  published  human 
factors  engineering  standards.  The  operator’s  position  at  the  terminal  was  determined 
from  photographs  made  of  four  operators  whose  height  ranged  from  56  inches  (142  cm)  to 
76  inches  (193  cm). 

Environmental  measures  were  obtained  in  the  correlation  processor,  communica¬ 
tions,  and  remote  display  system  vans.  Measures  consisted  of  temperature,  humidity,  and 
noise  levels.  The  wet  and  dry  bulb  temperatures  were  measured  with  a  Psychron,  Model 
566-2  psychrometer  (fahrenheit  readings).  Sound  levels  were  measured  with  a  sound  level 
meter,  Model  1565-B  (General  Radio),  which  conformed  to  ANSI  S1.4  standards  for  a 
Type  2  instrument.  The  meter  was  calibrated  with  a  Model  1567  sound  level  calibrator 
(General  Radio)  before  each  set  of  measurements.  Both  the  sound  levels  and  wet  and  dry 
bulb  temperatures  were  measured  in  the  same  locations  in  a  given  shelter.  Three  loca¬ 
tions  were  used  in  the  correlation  processor  and  two  in  the  other  shelters.  A  single  set  of 
measurements  was  taken  on  each  of  three  separate  days  during  the  summer  when  the 
single  function  testing  was  keeping  the  processors  operating.  While  the  shelters  were  not 
specifically  part  of  the  test,  the  environmental  conditions  there  might  approximate  what 
the  operators  would  experience  in  a  fielded  version  of  the  system.  It  was  also  known  that 
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computer  operators  had  complained  about  the  conditions  in  the  correlation  processor  dur¬ 
ing  earlier  tests,  so  it  was  desirable  to  get  environmental  measures  for  it. 

Data  Analysis 

Due  to  the  nature  of  the  data  collected,  extensive  analyses  did  not  seem  to  be  justi¬ 
fied.  Virtually  all  of  the  data  except  the  physical  measurements  of  the  terminal  and  the 
environmental  measures  were  completely  subjective.  Even  though  these  latter  measures 
are  completely  objective,  they  could  only  be  compared  with  published  standards  to 
determine  whether  the  system  met  these  standards.  The  analyses  conducted,  and  the 
decisions  made  concerning  interpretation,  are  described  below  for  each  type  of  data  col¬ 
lected. 

Questionnaires.  Descriptive  statistics  for  each  item  in  the  questionnaires  were 
prepared.  A  full  tabulation  including  frequency  distributions  is  presented  in  Appendix  E 
for  the  Human  Factors  Questionnaire  and  Appendix  F  for  the  CPX  Questionnaire.  To 
provide  meaning  to  these  data,  some  arbitrary  decisions  had  to  be  made.  By  consensus, 
the  research  staff  opted  for  the  following  interpretation  for  all  items  rated  on  the  "very 
inadequate"  to  "very  adequate"  scale: 

a.  Desirable  characteristics,  controls  or  functions  (adequate 
or  higher  ratings  by  80  percent  or  more  of  the  operators). 

b.  Suspect  characteristics,  controls  or  functions  (adequate  or 
higher  ratings  by  70-79  percent  of  the  operators). 

c.  Problem  characteristics,  controls  or  functions  (adequate  or 
higher  ratings  by  less  than  70  percent  of  the  operators). 

Two  other  types  of  data  were  to  be  collected  by  questionnaire.  Questions  on  poten¬ 
tial  safety  hazards  were  included  in  the  Human  Factors  Questionnaire.  However,  no 
hazards  were  reported.  Questions  on  physical  complaints  were  included  in  the  CPX 
Questionnaire.  These  were  simply  tabulated  for  each  day  of  the  exercise.  It  was  arbi¬ 
trarily  decided  that  any  complaint  developed  during  the  exercise  reported  by  10  percent 
or  more  of  the  operators  indicated  a  potential  problem. 

Interviews.  During  the  interviews  the  operators  were  given  considerable  latitude  in 
making  their  responses,  so  the  interviewer  and  another  member  of  the  research  staff 
coded  responses  to  the  questions  about  simple  and  complex  functions  using  the  scheme 
shown  in  Table  1.  Disagreements  in  coding,  though  very  few,  were  resolved  through 
discussion.  A  full  tabulation  of  the  response  codings  is  shown  in  Appendix  G. 

The  interviews  generated  a  large  number  of  spontaneous  comments  from  the  opera¬ 
tors.  These  were  recorded,  and  will  be  used  to  amplify  or  clarify  interpretations 
presented  in  the  Results  section. 

Direct  observation.  No  separate  analyses  of  notes  made  while  observing  the 
operators  was  conducted.  Items  from  these  notes,  like  those  from  the  interviews,  will  be 
used  to  justify  or  bolster  the  interpretations  of  other  data  presented  in  the  Results 
section. 
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TABLE  1 


Coding  Scheme  for  Responses  to  Items  1  through  78  of  the 
Human  Factors  Interview 


Question 

Value 

Meaning 

Have  you  used 

0 

No  response 

this  key? 

1 

Have  not  used 

2 

Have  used 

How  much  did 

0 

No  response 

you  use  it? 

1 

Very  little  (1-2  times) 

2 

Not  much  (3-5  times) 

3 

A  lot  (more  than  5  times) 

How  well  did 

0 

No  response 

it  work? 

1 

Unsatisfactorily 

2 

Satisfactorily 

3 

Well 

What  problems 

0 

No  comment 

did  you  have 

1 

Negative  comment 

with  it? 

2 

Neutral  comment 

3 

Positive  comment 

How  would  you 

0 

No  comment 

change  it? 

1 

Hardware  change 

2 

Software  change 

3 

Training  change 

4 

No  change 

Physical  measurements.  The  physical  measurements  of  the  hardware  and  the 
environmental  measurements  were  compared  with  applicable  human  factors  engineering 
standards  in  MIL  STD  1280, 19  MIL  STD  1472B,29  and  Van  Cott  and  Kinkade.21  No 
standards  could  be  found  for  some  of  the  measurements  obtained.  Deviations  from 
acceptable  ranges  were  noted,  and  interview  and  questionnaire  data  were  examined  to 
determine  what,  if  any,  comments  were  made  concerning  items  which  failed  to  meet  the 
standards.  The  data  from  all  of  these  sources  are  reported  and  discussed  together  in  the 
Results  section. 

Participants 

A  total  of  41  soldiers  were  trained  to  operate  the  JTFTB-A  terminals.  Shown  below 
(Table  2)  are  their  military  grades.  Also  shown  are  their  Military  Occupational  Specialties 
(MOSs)  (Table  3)  and  some  general  demographic  information  (Table  4).  It  can  be  seen  that 
their  military  grades  ranges  from  E2  through  02,  and  their  MOSs  were  in  the  intelligence 
or  intelligence-related  fields  (with  the  exception  of  the  lone  13F,  Fire  Support  Specialist). 
About  half  of  them  had  been  in  the  military  for  less  than  three  years  and  more  than  half 
had  been  working  in  their  MOS  for  less  than  two  years.  Most  were  under  30  years  of  age 
and  all  had  at  least  a  high  school  level  of  education. 

Results 

As  discussed  in  earlier,  the  human  factors  data  collection  effort  focused  on  those 
functions  and  characteristics  of  JTFTB-A  that  were  perceived  as  likely  to  be  included  in 
any  future  ASAS  hardware/software  system.  Obviously,  the  findings  regarding  these 
functions  and  characteristics  must  be  interpreted  in  light  of  the  situation  in  which  they 
were  obtained.  First  of  all,  it  must  be  remembered  that  no  clearly  defined  operational 
concept  or  set  of  missions  for  the  system  existed.  Hence,  operator  dissatisfactions  with 
or  perceived  shortcomings  of  the  system  may  have  resulted  from  unrealistic  expectations. 
Secondly,  the  level  of  operator  training  was  undoubtedly  below  what  would  be  expected 
for  a  fielded  system  in  an  operational  setting.  Not  all  of  the  35  operators  interviewed  had 
used  all  of  the  system's  functions,  and  in  many  cases  where  they  had,  they  had  done  so 
only  once  or  twice  (see  Appendix  G).  Therefore,  familiarity  with  the  system  was  less  than 
desired.  Finally,  since  the  system  was  intended  as  a  test  bed,  off-the-shelf  hardware 
items  were  employed  rather  than  items  designed  specifically  for  the  functions  incorpo¬ 
rated  in  the  system.  Nevertheless,  it  was  felt  that  operator  opinions  concerning  the 
system  as  a  whole,  the  functions  performed,  the  man/machine  interface,  and  the  working 
environment  would  provide  useful  information  for  designers  of  future  systems.  Therefore, 
the  discussion  of  the  results  is  oriented  along  these  lines  and  toward  this  end. 

*9US  Department  of  the  Army.  MIL-STD-1280:  Military  standard  keyboard 
arrangements.  Author,  January  1969. 

29US  Department  of  the  Army.  MIL-STD-1472B:  Human  engineering  design 
criteria  for  military  systems,  equipment  and  facilities.  Author,  December  1974. 

21h.  Van  Cott  tf  R.  G.  Kinkade  (eds).  Human  engineering  guide  to  equipment  desi 
(rev  ed).  Washington,  DC:  US  Government  Printing  Office,  19 
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TABLE  2 


Military  Grades 


Military 

grade 


Number  of 
operators 


E2 

E3 

E4 

E5 

E6 

W1 

W2 

W4 

01 

02 


3 

4 
9 
7 

11 

1 

1 

1 

3 

1 


TABLE  3 


Military  Occupational  Specialties  (MOS) 


MOS 


Number  of 
operators 


Enlisted  Personnel 


05G  2 

05H  1 

13F  1 

96B  19 

960  6 

97B  1 

98C  3 

98G  1 

Warrant  Officer 

964A  1 

973A  2 

Commissioned  Officer 

35A  2 

37  A  2 
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TABLE  4 

Demographic  Information 


Years 


Time  in  Military 

0-3 

3-6 

6-9 

9-12 

12-15 

More  than  15 


Number  of 
operators 


21 

6 

8 

3 

2 

1 


Amount  of  Experience  in  MOS 


1-2 

2-4 

4-6 

6-8 

8-10 

10-12 

12-14 


Age 


Less  than  20 

21-25 

26-30 

31-35 

36-40 

41-45 

46-50 


27 

6 

4 

3 

0 

0 

1 


7 

13 

16 

2 

2 

0 

1 


Highest  Level  of  Civilian  Education 


12 

13 

14 

15 

16 

More  than  16 


19 

7 

5 

1 

7 

2 


i 


5 

I 

*  . 

*: 


UL 


ll 


It  seemed  desirable  to  obtain  some  indication  of  how  well  a  fielded  system  modeled 
after  the  JTFTB-A  as  a  whole  would  serve  the  intelligence  analyst's  needs.  Therefore, 
some  items  in  the  questionnaire  were  designed  to  assess  global  impressions  of  the  system 
and  much  of  the  discussion  during  the  interviews  centered  on  this  subject.  Test  personnel 
also  contributed  many  comments  on  how  the  system  might  be  used.  Comments  were 
recorded  and  will  be  reproduced  in  the  following  subsections  as  appropriate. 

General  utility.  In  question  97  of  the  interview  (Would  this  system  help  or  hinder 
you  in  doing  your  present  job?),  54  percent  of  the  operators  made  comments  suggesting 
that  the  system  would  help  them,  while  26  percent  indicated  that  it  would  hinder  them. 
"Obtaining  the  desired  information  is  easy  if  the  operator  is  an  intelligence  analyst 
preferably  with  one  year's  experience,"  said  one.  Another  said,  "As  a  traffic  analyst  I  can 
see  where  the  enemy  is  and  what  he  is  trying  to  do.  I  can  associate  with  the  situation 
better  when  I  view  it  on  the  graphic  screen."  One  liked  the  direct  access  to  the  computer: 
"Entering  information  in  the  system  is  so  much  easier,  you  don't  have  to  fight  with  a 
bunch  of  people."  Another  said,  "The  operator  doesn't  have  to  bug  a  lot  of  people  for 
information,  it's  very  easy."  Others  liked  the  speed:  "It's  much  faster  than  manual  --  it 
takes  five  minutes  for  information  compared  to  two  or  three  days  manually."  Negative 
comments  questioned  the  appropriateness  of  this  particular  type  of  system  and  the 
difficulties  of  making  it  work  in  the  field.  Some  operators  were  concerned  that  they  did 
not  know  enough  to  use  it.  One  warrant  officer  said,  "We  need  somebody  in  the  exercise 
with  a  million  years  of  experience."  The  orange  battlefield  entity  hierarchy  (i.e.,  friendly 
units  acting  as  the  enemy)  caused  others  problems,  like  the  operator  who  could  not  work 
with  it  because  he  "did  not  know  the  range  of  our  own  artillery."  Others  could  not  p  :t  the 
information  they  wanted.  For  instance,  they  could  not  tell  if  a  unit  was  moving — they  got 
a  first  report  and  then  another  at  a  later  time  and  different  location — but  the  original 
was  still  on  the  screen.  They  did  not  know  if  it  was  the  same  unit  or  another  unit.  A 
warrant  officer  operator  was  concerned:  "The  ASAS  can't  be  operated  with  three  to  five 
terminals  —  that's  not  enough."  Similar  points  were  made  in  response  to  interview 
question  100  (What  is  your  overall  impression  of  the  system?),  but  only  37  percent  of  the 
comments  were  favorable  while  23  percent  were  negative. 

Opinions  on  the  system  were  generally  more  negative  during  the  CPX.  Responses  to 
the  CPX  Questionnaire  are  shown  in  Table  5.  The  first  column  for  each  day  in  the  table 
shows  the  percentage  of  respondents  who  gave  ratings  of  adequate  or  better  in  response  to 
each  item.  The  generally  lower  opinions  undoubtedly  resulted  from  the  difference  in  the 
situations  in  which  the  interview  and  questionnaire  data  were  obtained.  During  the  CPX 
the  operators  were  under  greater  stress  and  time  pressure  and  had  to  use  the  system  in  a 
much  more  realistic  manner  than  during  training.  As  can  be  seen  in  the  responses  to 
item  2,  ratings  of  adequate  or  better  ranged  from  only  26  percent  to  40  percent  for 
adequacy  of  system  operations.  Also,  system  accuracy  received  relatively  poor  ratings. 
Adequate  or  better  ratings  in  response  to  item  3  ranged  from  a  low  of  25  percent  to  a  high 
of  36  percent  of  the  operators.  However,  voice  communications  (item  6)  fared  much 
better,  receiving  adequate  or  better  ratings  from  75  percent  to  82  percent  of  the 
respondents. 

Adequacy  of  information  received.  The  interview  data  showed  that  the  operators 
were  generally  favorable  about  the  information  they  received.  In  response  to  interview 
question  79  (How  do  you  feel  about  the  adequacy  of  information  received?),  71  percent 
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TABLE  5 

Percent  of  Operators  Giving  Ratings  of  Adequate  or  Better, 
and  Mean  Ratings  of  Items  on  the  CPX  Questionnaire 


1 

Item 

% 

Day  1 

Mean 

% 

Day  2 

Mean 

% 

Day  3 
Mean 

% 

Day  4 
Mean 

,  *  ft 

1 .  How  adequate  is  the 

50 

3.40 

26 

2.68 

40 

3.20 

32 

3.14 

information  you  are 

»%v 

% 

receiving  from  the 

system? 

i- 

2.  How  adequate  is  the 

40 

3.20 

26 

2.63 

35 

2.95 

27 

2.73 

>.V. 

operation  of  the 

!  •  v 

system? 

3.  How  adequate  is  the 

30 

2.70 

31 

2.75 

25 

2.85 

36 

2.95 

• 

accuracy  of  the 

system? 

fv 

■V 

4.  How  adequate  is  the 

80 

4.20 

79 

4.11 

90 

4.10 

95 

4.09 

!W 

A/N  display? 

5 .  How  adequate  is  the 

60 

3.70 

39 

3.44 

55 

3.40 

59 

3.50 

F 

SIT  display? 

n 

6 .  How  adequate  is  the 

80 

4.00 

76 

4.12 

75 

3.90 

82 

3.86 

s 

voice  communica¬ 

1 

tions? 

as 

7 .  How  adequate  is  the 

30 

3.10 

32 

2.74 

24 

2.76 

20 

2.75 

S 

organization  of  the 

»“*  •* 

player  cell? 

i  * « 
.•  * 

£ 

8 .  How  adequate  is  the 

22 

2.22 

18 

2.00 

5 

1.90 

9 

1.77 

state  of  the  CPX? 

k 

Number  on  shift 

10 

19 

20 

22 

V> 
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made  positive  comments  while  23  percent  made  negative  comments.  Concern  about 
missing  information  was  expressed  by  many  operators  in  response  to  interview  question  81 
(How  do  you  feel  about  the  interpretation  of  missing  information?).  Forty-six  percent 
made  comments  indicating  that  they  could  deal  with  missing  information  compared  with 
23  percent  who  felt  they  could  not.  One  analyst  commented  "It  takes  experience  to  know 
when  information  is  missing.  At  times  when  airborne  [sensor]  platforms  are  not  in  the  air 
due  to  weather,  there  are  gaps  in  the  information."  One  operator  commented  that  things 
outside  the  JTFTB-A  would  affect  whether  the  system  worked:  "The  information  is  as 
adequate  as  the  sensors  that  give  it  to  them."  Considerable  concern  with  loss  of 
information  in  case  of  system  malfunction  was  expected.  However,  the  responses  to 
interview  question  88  (How  do  you  feel  about  the  transfer  of  information  to  a  manual 
TOC?)  did  not  bear  out  the  expectation.  Sixty-nine  percent  made  favorable  comments 
concerning  transfer  while  only  11  percent  made  negative  comments.  A  senior  analyst 
would  have  liked  to  have  had  a  copy  of  the  messages  that  went  into  the  system  to  keep 
going  if  it  had  problems:  "There  should  be  a  regular  dump  of  the  data  base."  This  same 
analyst  did  not  like  the  purge  function  because  "somebody  could  get  disgruntled  and  ruin 
the  data  base." 

Adequacy  of  the  information  received  was  rated  more  poorly  during  the  CPX.  On 
the  second  day  of  the  exercise  only  26  percent  of  the  operators  felt  that  it  was  at  least 
adequate  (item  1,  Table  5).  The  highest  rating  was  given  on  the  first  day.  However,  even 
at  the  beginning  of  the  exercise,  only  50  percent  of  the  operators  felt  the  information 
received  was  minimally  adequate. 

Response  time.  A  major  concern  throughout  the  test  was  expressed  in  responses  to 
interview  question  87  (How  do  you  feel  about  the  system  response  time  to  specific 
demands?).  Only  26  percent  of  the  operators  made  favorable  comments  while  57  percent 
made  unfavorable  ones.  The  major  complaint  concerned  the  time  it  took  to  redraw  the 
map  when  new  data  were  obtained.  A  modification  to  the  terminal  computer  cut  this 
time  from  30  to  10  seconds  during  training.  However,  when  a  bigger,  more  complex  map 
was  installed  for  the  CPX,  the  redraw  time  went  back  up  to  30  seconds.  Despite  the 
redraw  times,  observations  during  the  CPX  indicated  that  operators  actually  spent  very 
little  time  waiting  on  redraws.  Hence,  the  complaints  may  have  resulted  more  from 
minor  frustrations  than  actual  time  lost.  The  problem  appeared  to  be  greatest  when 
several  broad  range  present  queries  were  active  for  the  same  operator.  However,  this 
approach  was  considered  a  poor  use  of  the  system  by  most  operators. 

The  time  required  to  create  queries  bothered  some  operators.  One  stated  that  "the 
menus  are  very  repetitious  and  very  time  consuming."  A  senior  analyst  pointed  out, 
however,  that  most  queries  would  be  developed  in  advance  of  an  operation,  so  the  ongoing 
task  of  operators  would  be  to  make  minor  modifications  and  execute  stored  queries  rather 
than  create  new  ones. 

Problems  noted.  There  were  three  areas  frequently  mentioned  by  operators  as 
causing  problems  during  operations.  Each  of  these  will  be  discussed  briefly. 

(a)  Interference  among  functions.  All  but  one  of  the  operators  interviewed 
commented  that  functions  interfered  with  each  other  (interview  question  83,  "Does  the 
execution  of  one  function  interfere  with  the  execution  of  another  function?).  The 
consensus  was  summed  up  by  one  who  stated  very  simply  "You  can't  do  more  than  one 
function  at  a  time."  By  way  of  example,  this  operator  offered:  "While  creating  a  query, 
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you  must  CANCEL  the  function  and  go  back  to  DEFINE  AOI."  This  kind  of  interference 
undoubtedly  resulted  in  lost  time  as  well  as  being  a  source  of  frustration  for  the 
operators. 

(b)  Error  correction.  Typographical  errors,  if  noticed  immediately,  could  be 
easily  corrected  by  deleting  the  incorrect  information  and  entering  the  correct  informa¬ 
tion.  If  a  frame  was  already  completed  and  the  DONE  key  pressed,  then  the  usual 
procedure  was  to  CANCEL  the  function  and  try  it  again.  The  correction  of  typographical 
errors  with  functions  such  as  DELETE  CHARACTER  was  rated  as  operating  satisfactorily 
by  100  percent  of  the  operators.  The  CANCEL  function  also  was  rated  as  operating 
satisfactorily  by  100  percent  of  the  operators.  Many  of  the  errors  were  not  detected  by 
the  operators  until  the  information  wanted  did  not  appear  or  the  system  sent  an  error 
message.  Unfortunately,  the  questionnaire  and  interview  did  not  directly  ask  about  either 
of  these  situations.  Observation  of  the  operators  indicated  errors  and  the  system  response 
to  them  presented  a  continuing  problem  throughout  the  test.  There  were  several 
comments  indicating  the  kind  of  difficulties  the  operators  were  having.  The  EDR 
functions  had  some  local  error  detection  at  the  terminal:  "You  have  to  be  very  specific 
with  fill-in-the-blank  in  an  EDR  or  it  will  not  accept  what  you  write  in."  Other  errors 
were  transmitted  to  the  correlation  processor  which  would  send  a  message  back  to  the 
ALERT  QUEUE,  but  as  one  operator  put  it:  "The  ALERT  QUEUE  does  not  get  my 
attention."  These  messages  stay  in  the  queue  so  that  the  operators  had  difficulty 
determining  which  message  referred  to  what  function.  Moreover,  these  messages  from 
the  processor  did  not  give  much  information  about  why  there  was  an  error.  As  one 
warrant  officer  operator  put  it:  "The  error  messages  could  not  be  understood."  In  one 
case,  the  message  "ERROR  LOCATION  ERROR"  for  an  EDR  resulted  in  several  attempts 
to  change  the  information  in  the  location  field  based  on  the  advice  of  both  other  operators 
and  test  personnel.  The  actual  difficulty  was  the  scale  factor  for  the  distance  put  into 
the  field  LOCATION  ERROR. 

(c)  Organizational  factors.  Some  problems  observed  may  have  resulted  from 
the  manner  in  which  the  system  was  manned.  Obviously,  the  system  cannot  be  faulted  for 
inadequacies  in  the  operating  organization.  However,  it  is  difficult  to  pinpoint  exactly 
how  the  organization  might  have  affected  operations  and  impressions  of  system  adequacy. 
This  brief  discussion  is  included  here  only  to  show  that  the  personnel  felt  that 
organizational  changes  were  needed.  As  can  be  seen  in  the  response  to  item  7  of  the  CPX 
Questionnaire  (see  Table  5),  adequate  or  better  ratings  of  the  organization  of  the  player 
cell  ranged  from  a  high  of  32  percent  to  a  low  of  20  percent  during  the  four-day  exercise. 
The  problem  was  well  stated  by  one  of  the  test  officers:  "We  tried  to  drop  a  system  into 
an  organization  -  we  should  identify  the  resources  needed  and  develop  the  organization 
from  that." 

Terminal  Functions 

The  dialogue  between  the  operator  and  the  system  is  carried  on  through  the 
terminal.  Hence,  the  terminal  is  really  the  heart  of  the  system  for  the  operator.  What  he 
or  she  can  do  with  the  system  is  limited  by  what  information  the  terminal  can  provide  and 
what  functions  it  can  perform,  i.e.,  the  dialogue  characteristics.  Therefore,  the  ratings  of 
terminal  functions  are  really  ratings  of  the  dialogue. 

Display  of  information.  The  alphanumeric  display  was  rated  on  seven  information 
characteristics  with  six  of  these  receiving  adequate  ratings  from  most  operators  with  a 
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range  of  78  percent  to  84  percent  (Table  6).  The  other  characteristic,  the  incoming 
message  indicator,  was  rated  adequate  by  only  65  percent  of  the  operators.  It  signaled 
there  was  a  new  message  in  the  operator's  queue  by  incrementing  a  message  counter 
displayed  on  the  second  line  of  the  display.  It  was  easily  overlooked,  as  indicated  by  one 
operator  who  said:  "I  would  like  for  the  ALERT  QUEUE  to  have  an  indicator  like 
ACKNOWLEDGE  NEW  DATA  [i.e.,  flashing." 

Twelve  of  the  13  information  characteristics  of  the  situation  [graphic]  display  were 
rated  adequate  by  most  operators  with  a  range  of  81  percent  to  95  percent  (Table  6).  The 
exception  was  location  of  the  cursor,  which  was  rated  adequate  by  only  64  percent.  This 
lower  percentage  probably  resulted  because  the  cursor  could  be  lost  from  the  screen  and 
because  it  moved  in  abrupt  jumps  instead  of  smoothly. 

The  alphanumeric  display  was  also  rated  quite  highly  on  the  CPX  Questionnaire, 
with  ratings  of  adequacy  or  better  ranging  from  79  percent  to  95  percent  on  the  four  days 
of  the  exercise  (see  item  4,  Table  5).  The  situation  display  did  not  fare  as  well  (item  5, 
Table  5).  Adequate  or  better  ratings  ranged  from  a  high  of  60  percent  to  a  low  of  39 
percent  during  the  CPX.  Respondents  who  gave  poor  ratings  were  not  asked  why  the 
display  was  not  adequate,  so  the  reasons  cannot  be  known  for  sure.  However,  the 
difference  in  the  situations  in  which  the  Human  Factors  Questionnaire  and  the  CPX 
Questionnaire  were  administered  probably  accounts  for  the  difference.  During  the  CPX, 
the  system  was  being  used  in  a  more  realistic  manner.  Shortcomings  not  noticed  during 
training  may  have  come  to  light. 

Simple  functions.  The  operators  were  asked  about  43  simple  functions  on  the 
terminal  during  the  interview.  The  results  are  summarized  in  Table  7.  For  35  of  these 
functions,  over  90  percent  of  the  operators  reprted  they  worked  satisfactorily.  An 
additional  five  functions  were  reported  as  working  satisfactorily  by  more  than  80  percent. 
Of  the  remaining  three  functions,  two — DIAGNOSTICS  and  INPUT  MAP— were  never  used 
by  an  operator.  The  third,  FAST  MODE,  was  rated  acceptable  by  76  percent.  The  largest 
number  of  positive  comments  (21)  was  received  for  the  numeric  key  cluster.  The  next 
largest  number  was  8  for  the  cursor  control  keys.  The  largest  number  of  negative 
comments  (24)  was  received  for  the  DISPLAY  ALPHA  function.  These  negative 
comments  mostly  concerned  the  difficulty  of  telling  which  display  was  attached  to  the 
keyboard.  One  operator  observed:  "There  should  be  a  light  under  each  screen  to  indicate 
which  one  is  on."  The  next  largest  number  of  negative  comments  (18)  was  shared  by 
ACKNOWLEDGE  NEW  DATA  and  the  VOICE  MODE  switch.  The  ACKNOWLEDGE  NEW 
DATA  key  had  to  be  used  whenever  a  present  query  updated  the  situation  display  to  stop 
the  new  symbols  from  flashing.  This  would  interrupt  any  other  functions  in  progress.  "I 
was  locked  out  of  the  system  for  20  minutes  waiting  while  new  data  was  coming  in," 
complained  one  operator.  Part  of  the  problem  may  have  been  due  to  "bugs"  in  the 
routines  that  handled  this  function,  as  indicated  by  the  operator  who  reported  that 
characters  deleted  from  the  situation  display  reappeared  whenever  he  pressed  the 
ACKNOWLEDGE  NEW  DATA  key.  There  was  a  general  dislike  for  the  way  the  voice 
communication  system  worked.  As  one  operator  put  it:  "I  would  like  for  it  to  operate  like 
a  telephone." 

Complex  functions.  All  35  of  the  complex  functions  were  reported  as  operating 
satisfactorily  by  80  percent  or  more  of  the  operators  in  the  interview  (Table  8). 
Thirty-three  of  these  were  rated  as  satisfactory  by  over  90  percent.  The  largest  number 
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TABLE  6 


Ratings  of  Information  Characteristics  of  Displays 
(from  Human  Factors  Questionnaire) 


Alphanumeric  display-information  characteristics 

Number  Ratings  of  adequate  Mean 

responding  or  better  rating 

Number  Percent 


System  status 

37 

31 

84 

3.81 

Attention-getting  power 

37 

31 

84 

3.97 

of  blink 

Execution  completed 

36 

30 

83 

4.00 

Prompts 

37 

30 

81 

4.00 

Execution  underway 

37 

30 

81 

3.97 

Error  messages 

37 

29 

78 

3.95 

Incoming  message 

37 

24 

65 

3.51 

Graphic  display— 

-information  characteristics 

Unknown,  enemy  and/or 

37 

35 

95 

4.19 

friendly  entities 

Map  scale 

37 

35 

95 

4.19 

Time  of  last  setting 

36 

34 

94 

4.19 

Type  of  entity 

37 

33 

89 

4.08 

Targetability 

37 

32 

86 

4.08 

Control  measures 

37 

32 

86 

4.08 

Map  location 

37 

32 

86 

4.05 

Map  width 

36 

3.1 

86 

4.08 

Attention-getting  power 

35 

30 

86 

4.14 

of  blink 

Map  features 

38 

31 

82 

3.95 

Location 

37 

30 

81 

3.95 

Echelon 

36 

29 

81 

3.94 

Location  of  cursor 

36 

23 

64 

3.58 

TABLE  7 


Frequency  of  Use,  Operator  Satisfaction,  and  Comments  on  Simple  Functions  From 
Human  Factors  Interview  (arranged  in  descending  order  based  upon  Percent  of 
operators  indicating  that  they  used  a  function  five  or  more  times) 


Function 


Number  Times  Used 
1-2  3-5  5+ 


%  Satis¬ 
factory 


Comments 
Neut  Pos 


Simple  Functions 


Display  graphic 

35 

35 

Display  alpha 

35 

35 

Enter 

35 

35 

Done 

35 

35 

Cancel 

35 

35 

Acknowledge  new  data 

35 

35 

Select  map  scale 

35 

35 

Select  map  center 

35 

35 

Home 

35 

34 

Numeric  key  cluster 

35 

35 

Clear 

35 

35 

Cursor  control  keys 

35 

35 

Tab 

33 

32 

Log  off 

35 

31 

Scroll  up 

34 

31 

Prepare  free  text  message 

35 

31 

Voice  mode  switch 

35 

32 

Scroll  down 

34 

30 

Move  graphic 

32 

28 

Write  graphic 

32 

27 

Delete  character 

25 

19 

Color  keys 

31 

30 

Plotter 

15 

12 

Insert  character 

25 

20 

Normal  character 

27 

20 

2X  character 

28 

22 

Fast  mode 

25 

18 

Retransmit  message 

13 

8 

Erase  line 

18 

9 

Left  front  line 

7 

2 

Erase  page 

12 

6 

Delete  line 

24 

14 

Normal  background 

16 

3 

Reversed  background 

16 

3 

Right  front  line  trace 

7 

2 

Insert  line 

22 

22 

Execute  vector 

4 

0 

End  vector 

4 

o 

Diagnostics 

0 

0 

Front  line  trace  off 

5 

0 

Normal  intensity 

8 

2 

Low  intensity 

8 

2 

Input  map 

0 

0 

35 

35 

35 

35 

35 

34 

34 

34 

33 

33 

32 

31 

29 

24 

21 

20 

20 

19 

16 

15 

10 

9 

9 

8 

8 

8 

5 

4 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 


97 

94 

100 

100 

100 

100 

97 

100 

94 

97 

94 

91 
100 
100 

97 

100 

100 

97 

97 

100 

100 

100 

80 

100 

96 

100 

76 

92 
89 

100 

92 

100 

100 

100 

100 

100 

100 

100 

0 

80 

88 

88 

0 


4 

24 

10 

9 

4 

18 

8 

2 

13 

3 
8 
9 

4 
2 
3 

5 

18 

2 

2 

7 

0 

11 

13 

1 

2 

9 

16 

1 

2 

1 

0 

0 

1 

2 

1 

0 

0 

0 

0 

0 

1 

1 

0 


30  0 

11  0 

21  4 

21  5 

25  6 

13  4 

25  2 

33  0 

16  6 

11  21 

26  1 

18  8 

28  1 

29  4 

30  1 

26  4 

13  4 

30  2 

26  0 

24  1 

25  0 

18  2 

6  2 

22  2 

25  0 

19  0 

7  2 

12  1 

15  1 

6  0 

11  1 

23  1 

14  1 

14  0 

6  0 

21  1 

4  0 

4  0 

0  0 

4  0 

7  0 

7  0 

0  0 


of  positive  comments  was  given  to  OFFSETTING  DISPLAY  SYMBOL.  It  also  received  six 
negative  comments.  The  operators  found  that  offsetting  the  symbols  helped  when  the 
screen  was  crowded  with  entities:  "This  was  helpful  to  make  sense  out  of  a  cluttered 
screen."  The  difficulty  with  the  function  was  the  time  it  took  to  mark  the  symbol  and 
then  mark  the  offset  position:  "The  screen  gets  cluttered  quickly  and  the  offset  process  is 
very  tedious." 

Two  complex  functions  received  eight  positive  comments  in  the  interview  - 
MODIFY  QUERY  and  CREATE  PAST  QUERY.  The  MODIFY  QUERY  also  received  six 
negative  comments,  while  the  create  function  received  nine  negative  ones.  The  MODIFY 
QUERY  function  allowed  an  operator  to  change  a  query  that  was  already  constructed 
instead  of  building  one  from  scratch.  This  allowed  the  use  of  another  operator's  queries. 
It  also  allowed  an  operator  to  store  a  set  of  partially  completed  queries  which  could  be 
modified  to  fit  a  specific  situation.  The  MODIFY  QUERY  was  ar  easy  function  to  use: 
"The  functions  are  so  consistent  that  I  have  memorized  what  the  fra.  as  are  in  the  menus 
and  I  can  press  DONE  until  the  frame  in  which  I  am  interested  tppears."  The  query 
creation  functions  were  liked  for  their  straightforward  menus:  "It's  easy  when  you  follow 
the  prompts  to  create  a  query."  The  negative  comments  came  because  the  process  could 
take  a  long  time  and  because  there  could  be  up  to  64  menus  for. selection:  "Some  menus 
repeat  themselves  too  much."  The  past  query  function  was  seen  as  one  of  the  most  useful 
information  gathering  tools  as  the  exercise  developed.  "A  Past  Query  was  helpful  in 
making  a  record  of  what  has  happened  in  the  past  four  to  six  hours." 

The  dialogue  required  in  constructing  queries  caused  some  problems.  Measures 
could  not  always  be  put  into  the  forms  in  the  way  the  operators  were  accustomed.  It  was 
pointed  out  that  24  hours  had  to  be  entered  as  1  day,  00  hours,  and  north  could  not  be 
entered  as  360  degrees  but  had  to  be  0  degrees.  When  a  query  was  started  on  a  terminal 
that  subsequently  went  down,  it  could  not  be  deactivated  by  any  other  terminal.  A 
warrant  officer  operator  said  he  "would  like  an  ID  on  the  [displayed!  symbols  to  match  the 
A/N  [alphanumeric]  printout."  According  to  a  test  officer,  "present  queries  are  really 
that  efficient  in  that  they  slow  the  machine  down  if  there  are  a  whole  bunch  [100  or  mo/e] 
that  are  active."  He  explained  that  this  happens  because  each  time  a  sensor  report  comes 
in  it  has  to  be  matched  against  each  active  present  query.  The  system  is  more  efficient  if 
an  operator  simply  writes  a  series  of  past  queries  and  periodically  activates  several  of 
them  to  see  if  anything  new  has  come  up.  The  only  way  to  review  a  query  was  to  use  the 
MODIFY  function.  As  an  improvement,  "you  need  a  summary  of  a  query  so  you  can  tell 
something  about  it  when  you  look  at  someone  else's  query." 

The  largest  number  of  negative  comments  (19)  in  the  interview  was  made  about  the 
DISPLAY  QUEUE  ENTRIES  function.  The  operators  found  the  queue  procedures  difficult 
to  use,  and  some  remarked  they  had  missed  messages  or  accidentally  deleted  them.  As 
one  said,  "Once  there  were  24  messages  in  my  OPERATOR  QUEUE  before  I  realized  they 
were  there."  Another  said,  "The  ALERT  QUEUE  does  not  get  my  attention."  One 
suggested:  "There  should  be  a  menu  that  comes  up  when  displaying  queue  entry,  allowing 
[the  operator]  to  review  or  modify  [the  message]." 

The  next  largest  number  of  negative  comments  (15)  were  made  about  inputting 
control  measures  with  the  joystick.  The  major  complaints  were  that  it  was  a  tedious, 
inaccurate  process:  "The  process  was  not  too  precise  -  I  have  all  the  coordinates  for 
FEBA  [Forward  Edge  of  the  Battle  Area],  but  can't  remember  where  it  was."  Another 
operator  said,  "Control  boundaries  are  extremely  hard  to  draw,  they  need  to  be 
simplified."  There  was  also  concern  that  the  color  coding  was  not  consistent  with 
standard  military  maps:  "The  control  boundaries  were  green  -  should  be  blue." 
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TABLE  8 


Frequency  of  Use,  Operator  Satisfaction,  and  Comments  on  Complex  Functions  From 
Human  Factors  Interview  (arranged  in  descending  order  based  upon  Percent  of 
operators  indicating  that  they  used  a  function  five  or  more  times) 


Function 


Number  Times  Used  %  Satis- 
1-2  3-5  5+  factory 


Comments 
Neut  Pos 


Complex  Functions 

Correcting  typo 
Storing,  retrieving,  or 
modifying  AOI 
Executing  stored  query 
Reducing  symbology  on  sit 
display 

Displaying  queue  entries 
Retrieving  control  measure 
from  data  base 
Restoring  symbology  on  sit 
display 

Modifying  stored  query 
Creating  EDR 
Reading  location  of  cursor 
Clearing  all  or  part  of 
current  display 
Creating  present  query 
Inhibiting  or  restoring  unit 
symbols  on  sit  display 
Offsetting  displayed  symbol 
Dropping  queue  entries 
Initializing 

Displaying  EDR  or  FDR  on 
A/N  display 

Manipulating  A/N  display 
Modifying  EDR  and  FDR 
Using  logical  zoom 
Creating  past  query 
Purging  control  measure 
from  data  base 
Inputting  control  measure 
with  joystick 
Purging  data  EDR  and/or 
FDR  from  data  base  and 
sit  display 

Creating  aggregation 
Sending  sit  display 
Sending  A/N  display 
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Function 

Number  Times  Used 

%  Satis¬ 

Comments 

1-2 

3-5 

5+ 

factory 

Neg 

Neut 

Pos 

(  s"  * 

Displaying  FDR  on  sit 
display 

22 

15 

10 

95 

5 

19 

0 

1--. .-. 

1-'  V 

Modifying  aggregation 

24 

16 

7 

96 

3 

22 

0 

L 

Updating  target  status 

26 

14 

7 

100 

5 

25 

o 

Creating  future  query 

25 

13 

5 

96 

2 

22 

1 

L/\r 

Creating  and  sending  cue 
photo  message 

26 

14 

5 

96 

7 

21 

0 

/  V 

k* .  .  \ 

Creating  FDR 

27 

10 

4 

93 

8 

18 

2 

■,t*,  ts 

"Xy 

Creating  and  sending  cue 
TACELINT  message 

18 

8 

3 

94 

5 

13 

1 

»y*\ 

Inputting  control  measure 
with  graph  tablet 

3 

2 

0 

100 

1 

5 

0 

» V  V  . 
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Two  complex  functions  received  13  negative  comments  each:  CREATE  EDR  and 
AGGREGATION.  Most  of  the  complaints  about  dialogue  had  to  do  with  these  two 
functions.  The  EDR  function  used  a  mixed  dialogue  that  consisted  of  some  menus 
followed  by  a  form  filling  frame.  "I  thought  it  was  too  time  consuming  -  the  menus  need 
to  be  consolidated,”  said  one  operator  and  another  found  it  was  necessary  ”to  be  very 
specific  with  fill-in-the-blanks  or  it  would  not  accept  information.”  The  form  of 
numbers,  placement  of  decimal  points,  and  scale  factors  were  among  the  things  that  had 
to  be  watched.  There  were  several  characteristics  of  entities  that  operators  would  like  to 
have  had  in  the  EDR  that  were  not  there.  One  suggested  "adding  an  annotation  block  to 
the  EDR”  to  share  information  with  other  operators.  The  AGGREGATION  function  was 
not  difficult  but  it  was  tedious.  As  one  operator  said,  "It  was  easy  when  I  read  the 
prompts."  Since  each  child  had  to  be  aggregated  separately,  it  was  easy  to  make  a 
mistake  in  the  middle  of  a  series  of  entities:  "I  was  unable  to  do  this  correctly."  This 
function  had  problems  in  the  early  parts  of  the  test,  so  that  some  operators  may  have 
been  confused  about  how  it  worked  by  the  time  the  CPX  was  held.  Two  of  the  operators 
who  were  asked  to  make  aggregations  by  the  test  team  during  the  CPX  did  not  remember 
how  to  do  it. 

Terminal  Design 

Design  of  military  equipment  to  meet  human  factors  needs  of  personnel  is  governed 
by  MIL-STD-1472B.22  This  document  compiles  relevant  human  factors  information  into  a 
series  of  tables  and  charts  to  cover  all  of  the  types  of  man-machine  systems  being 
acquired  by  the  Department  of  Defense  at  the  time  of  its  publication.  An  additional 
standard,  MIL-STD- 1280,23  specifically  covers  the  design  of  keyboards.  The  STANDARD 
in  effect  at  the  time  this  research  was  done  had  a  section  on  consoles,  but  the 
recommendations  were  based  mainly  on  research  with  radar  displays  and  discrete  control 
systems  (e.g.,  knobs,  dials,  and  switches).  Because  video  display  terminals  are  used  in  a 
qualitatively  different  way  from  these  early  types  of  consoles,  many  of  the  features  of 
the  JTFTB-A  ter  mined  are  not  covered  by  the  standards. 

In  general,  the  terminal  provided  a  workable  user-computer  interface  with  the 
JTFTB-A  system.  It  had  many  problems  that  made  the  operator  uncomfortable,  made 
some  tasks  more  difficult  to  execute  than  they  might  have  been,  and  made  some  functions 
fairly  difficult  to  learn.  There  was,  however,  no  evidence  that  any  feature  of  the 
terminal  would  prevent  an  operator  from  carrying  out  any  of  the  intelligence  tasks  so  far 
identified  in  the  documentation  or  by  the  test  participants. 

Keyboard.  A  view  of  the  terminal  keyboard  is  shown  in  Figure  1.  The  scale  in  the 
photograph  is  m  centimeters.  Questionnaire  results  are  summarized  in  Table  9.  Over  80 
percent  of  the  operators  gave  adequate  ratings  to  the  terminal  for  size  of  labels,  control 
type,  control  shape,  correct  labels,  size  of  controls,  understandable  labels,  location  of 
labels,  and  reach  distance  to  noncritical  controls.  All  of  these  features  also  met  the 
applicable  standards.  Lower  ratings  were  given  to  critical  control  location  with  only  41 
percent  of  the  operators  rating  it  as  adequate.  The  location  of  controls  on  the  function 


22us  Department  of  the  Army,  op.  cit.,  1974 
23us  Department  of  the  Army,  op.  cit.,  1969. 
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Figure  1 .  JTFTB-A  terminal  keyboard 


TABLE  9 


Summary  of  Control  Layout  Responses 
From  Human  Factors  Questionnaire 


Number  Rated  Adequate  Mean 

Item  Responding  Number  Percent  Rating 


Control  layout 


Size  of  labels 

38 

37 

97 

4.29 

Control  type 

37 

34 

92 

3.95 

Shape 

38 

34 

89 

4.05 

Correct  labels 

38 

34 

89 

4.13 

Size 

38 

32 

84 

4.00 

Understandable  labels 

37 

31 

84 

4.05 

Location  of  labels 

38 

31 

82 

3.92 

Noncritical  control  reach  distance 

37 

30 

81 

3.81 

Control  visibility 

37 

29 

78 

3.95 

Function  grouping 

37 

29 

78 

3.76 

Absence  of  unrelated  or  confused  markings 

36 

27 

75 

3.89 

Noncritical  control  location 

37 

27 

73 

3.68 

Spacing  between  controls 

38 

26 

68 

3.68 

Control  angle  of  view 

37 

25 

68 

3.73 

Keyboard  feel 

38 

24 

63 

3.61 

Critical  control  reach  distance 

37 

19 

51 

3.11 

Critical  control  location 

37 

15 

41 

3.00 

keyboard  was  not  liked  by  several  operators.  Of  particular  concern  was  the  location  of 
two  frequently  used  controls,  ENTER  and  DONE,  on  the  top  row  of  the  keyboard.  Thirty 
of  the  35  operators  suggested  changes  in  the  location  of  controls  when  interviewed.  One 
even  provided  the  test  team  with  a  drawing  for  a  new  keyboard.  Some  examples  of  the 
suggested  changes  were  "PICK,  TAB,  and  CLEAR  should  be  in  the  ADMIN  section"} 
"Relocate  ADMIN  section  closer  to  numeric  pad";  and  Relocate  TAB  on  A/N  keyboard  like 
regular  typewriter." 

Critical  control  reach  distance  was  rated  as  adequate  by  51  percent  of  the 
operators.  The  location  of  the  joystick  and  the  top  three  rows  of  function  keys  are  beyond 
the  reach  of  a  5th  percentile  male  in  a  sitting  position  as  required  by  the  standard.  This 
includes  the  ENTER  and  DONE  keys.  The  reach  distance  was  calculated  from  the 
photographs  and  then  applied  to  the  chart  on  page  530  of  Van  Cott  and  Kinkade.24  The 
distance  to  the  joystick  is  105  cm  while  the  reach  range  at  a  30  degree  angle  is  76.2  cm. 
"There  is  lots  of  reaching  to  select  items  [using  the  keyboarcO,"  commented  one  operator 
and  another  said,  "The  joystick  is  to  far  away;  it’s  awkward  to  reach.  Move  it  nearer  the 
function  keyboard."  These  controls  can  be  reached  by  leaning  over  the  keyboard  or  by 
moving  closer  to  the  terminal,  but  both  solutions  make  it  difficult  to  see  thr  situation 
display  at  the  operator's  right.  Moving  the  operator  closer  to  the  screen  is  hampered  by 
the  protruding  keyboard  shelf  and  the  limited  leg  room  under  the  console.  Leaning  over 
also  has  the  disadvantage  of  making  the  operator  uncomfortable  and  could  be  fatiguing 
over  an  extended  period  of  time. 

The  alphanumeric  keyboard  conformed  to  MIL-STD-1280  except  for  the  operation  of 
the  shift  key.  However,  the  flat-topped  function  keys  are  contrary  to  the  recommenda¬ 
tions  of  the  standard.  Also,  the  power-on  light  looks  like  a  key,  and  it  is  white  instead  of 
the  required  green. 

Workspace.  Operator  ratings  of  workspace,  summarized  in  Table  10,  were  generally 
adequate.  However,  test  personnel  observed  that  the  terminal  keyboard  and  writing 

TABLE  10 

Summary  of  Workspace  Ratings 
From  Human  Factors  Questionnaire 


Ratings  of 


Item 

Number 

responding 

adequate  of  better 
Number  Percent 

Mean 

rating 

Leg  room 

38 

37 

97 

4.24 

Seating  cushion 

34 

31 

91 

Seating  backrest 

35 

30 

86 

nWM 

Elbow  room 

38 

32 

84 

'K.ILLJ 

Seating  horizontal  adjustment 

34 

28 

82 

3.94 

Seating  vertical  adjustment 

35 

27 

77 

3.74 

Storage  room 

37 

35 

2.92 

24yan  Cott  if  Kinkade,  op.  cit.,  1972. 
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surface  were  too  high  above  the  floor  for  short  operators  unless  special  chairs  were 
provided.  The  standard  requires  that  both  of  these  should  be  approximately  65  cm  above  the 
floor  to  accommodate  the  5th  percentile  male  using  a  standard  adjustable  chair.  The 
JTFTB-A  surfaces  were  84  cm  above  the  floor,  which  meant  short  operators  worked  either 
with  their  arms  at  shoulder  level,  or,  if  they  were  able  to  adjust  the  chair,  with  their  feet 
dangling  in  the  air.  No  direct  rating  of  this  aspect  of  the  terminal  was  made  and  77  percent 
of  the  operators  rated  the  vertical  seating  adjustment  as  adequate.  One  smaller  operator 
remarked,  ’'There  needs  to  be  a  foot  rest  for  short  operators."  Other  comments  in  the 
interviews  were:  "It’s  too  high  in  the  air  -  get  it  at  typing  level,"  and  "The  chair  should  be 
adjustable  so  the  operator's  feet  don't  dangle." 

Various  other  dimensions  of  the  terminal  were  found  either  in  conflict  with  the 
standard  or  disliked  by  the  operators.  The  width  of  the  writing  area  is  54  cm,  which  is 
less  than  the  61  cm  minimum  recommended  by  the  standard.  Eight  operators  complained 
that  the  terminal  was  difficult  for  lefthanded  persons  to  use:  "As  a  lefthander,  I  find  the 
joystick  very  awkward  [to  use]."  The  space  under  the  keyboard  and  writing  area  is 
insufficient  for  large  operators  because  there  is  only  42.4  cm  clearance  from  the  keyboard 
lip  to  the  panel  while  the  standard  requires  a  minimum  of  46  cm.  One  operator  found  this 
a  problem:  "I  knock  my  knees  under  the  A/N  console  when  I  stand  up."  The  standard  also 
requires  a  kick-hole  clearance  of  10  cm  in  the  vertical  and  horizontal  dimension,  but  the 
terminal  has  one  that  measures  14.9  cm  high  by  7.0  cm  deep.  This  probably  caused  one 
operator  to  complain,  "I  always  scrape  my  boots  on  the  console." 

Joystick.  In  addition  to  the  problem  of  locating  the  cursor,  the  operation  of  the 
joystick  itself  was  rated  down  by  many  operators.  Control  of  the  cursor  for  the  situation 
display,  which  was  done  mainly  with  the  joystick,  was  rated  adequate  or  better  by  only  33 
percent  of  the  operators.  When  asked  to  comment  about  the  joystick  in  the  interview,  77 
percent  of  the  operators  made  comments  that  were  classified  as  negative  while  only  nine 
percent  made  positive  ones.  The  difficulties  came  from  two  sources.  First,  many  of  the 
operators  were  unaware  that  the  MOVE  GRAPHIC  function  put  the  joystick  into  a  pixel 
move  mode  which  caused  the  cursor  to  move  in  small  steps.  Only  nine  of  the  operators 
did  report  knowing  about  this.  Those  who  did  know  were  using  it  in  a  character- move 
mode  which  caused  it  to  take  large  jumps.  Second,  the  cursor  was  difficult  to  control  for 
fine  movements.  The  rate-control  design  had  a  very  steep  slope  which  meant  that  the 
cursor  would  move  very  little  or  not  at  all  with  light  pressure,  while  a  slight  increase 
would  send  it  racing  for  the  edge  of  the  screen.  This  caused  difficulties:  "I  lost  the 
cursor  when  moving  it  with  the  joystick."  Sometimes  the  cursor  would  drift  because  the 
control  did  not  return  to  a  zero  position:  "...and  even  the  thumbwheels  on  the  joystick 
wouldn't  adjust  it."  Another  operator  commented:  "The  joystick  is  the  pits  -  I  would 
prefer  a  [track!  ball  control."  Perhaps  the  experience  with  video  arcade  games  had  made 
today's  a  soldier  a  critical  judge  of  position  controls. 

Alphanumeric  display.  Four  of  the  seven  characteristics  of  the  alphanumeric  display 
were  rated  adequate  or  better  by  at  least  80  percent  of  the  operators  (Table  11).  Only  65 
percent  rated  the  absence  of  glare  as  adequate,  however.  The  flat  glass  shields  placed  in 
front  of  each  screen  to  prevent  classified  information  from  being  compromised  was  the 
location  of  the  glare.  Because  the  displays  are  tilted  back  at  an  angle  of  11  degrees  from 
vertical,  they  pick  up  reflections  from  the  overhead  lights.  Operators  commented:  "The 
glare  was  bad  -  it  was  easy  to  get  eyestrain,"  and  "There  was  too  much  glare.  I  had  a 
headache  all  the  time  after  using  it."  Absence  of  flicker  was  rated  as  adequate  by  78 
percent  of  the  operators  and  no  comments  were  made  about  this.  The  alphanumeric 
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TABLE  11 


Ratings  of  Display  Characteristics 
(from  Human  Factors  Questionnaire) 


Alphanumeric  display-display  characteristics 


Item 

Number 

responding 

Ratings  of  adequate 
or  better 

Number  Percent 

Mean 

rating 

K*1\\ 

C*%‘* 

s:;: 

V.-' 

r  -  *■- 

me 

Display  brightness 

37 

36 

97 

4.24 

m 

Viewing  distances 

37 

32 

86 

4.08 

v*v 

Legibility 

37 

31 

84 

4.11 

:# 

Angle  of  view 

37 

30 

81 

3.95 

vS> 

Absence  of  flicker 

37 

29 

78 

4.00 

-  A. 

*  o 

l*  -  * 

Control  of  cursor 

35 

24 

69 

3.71 

“ir 

*>*:*■: 

Absence  of  glare 

37 

24 

65 

3.68 

.  ■  .  " 

Graphic  display— 

-display  characteristics 

^  .  ■ 

Symbol  clarity 

36 

35 

97 

4.22 

Color  differences  identity 

36 

32 

89 

4.08 

VI I  l 

Display  brightness 

36 

31 

86 

4.06 

_  •  _  • 

Intensity  differences 

36 

31 

86 

4.08 

Absence  of  flicker 

36 

29 

81 

3.92 

.*-V; 

Legibility 

37 

30 

81 

3.95 

Angle  of  view 

36 

27 

75 

3.83 

id 

Absence  of  glare 

37 

22 

59 

3.49 

RS 

Control  of  cursor 

36 

12 

33 

2.92 

27 


display  had  a  full  range  of  controls  (e.g.,  brightness,  contrast,  focus)  for  the  operator  to 
adjust  as  required  by  the  standard. 

The  angle  of  view  was  rated  adequate  by  81  percent  of  the  operators,  but  at  a 
viewing  distance  of  75  cm  the  alphanumeric  screen  presents  the  5th  percentile  male  with 
viewing  angles  of  0.0  to  15.6  degrees,  which  is  above  the  preferred  range  of  -30  to  0  for 
critical  displays.  A  visiting  general  commented:  "The  small  print  on  the  A/N  display  is 
hard  to  read  with  bifocals  [without  tilting  my  head  back],"  while  an  operator  said,  "I 
always  have  to  tilt  my  head  to  see  the  OPERATOR  QUEUE."  The  character  size  on  the 
alphanumeric  display  is  2.82  mm  high  by  2.00  mm  wide,  which  presents  an  angle  of  view  at 
a  75  cm  viewing  distance  of  less  than  the  15  seconds  of  arc  required  by  the  standard. 
Most  operators  did  not  find  this  to  be  a  problem,  as  86  percent  rated  the  viewing  distance 
and  84  percent  rated  the  legibility  of  the  display  as  minimally  acceptable.  There  were 
some  complaints  in  the  interviews  about  the  small  character  size,  however.  Four 
operators  told  the  interviewer  things  like  "The  letters  on  the  A/N  display  are  too  small." 

Situation  display.  The  ratings  of  the  situation  or  graphic  display  were  generally  at 
least  adequate  including  symbol  clarity,  color  differences  identity,  display  brightness, 
intensity  differences,  absence  of  flicker,  and  legibility.  Only  two  characteristics,  absence 
of  glare  and  viewing  angle,  were  rated  adequate  by  fewer  than  80  percent  of  the 
operators.  The  source  of  glare  is  the  same  as  the  alphanumeric  display  and  the  comments 
were  similar.  Angle  of  view  was  rated  adequate  by  75  percent  of  the  operators.  The 
vertical  viewing  angle  was  similar  to  the  alphanumeric  display  because  the  center  lines 
are  at  the  same  height.  The  -4.2  to  19.6  degree  angle  of  the  situation  display  falls  outside 
the  recommended  range.  The  operator  usually  sits  on  the  left  side  of  the  terminal  to 
operate  the  keyboard  which  puts  the  display  at  a  30  degree  angle  to  the  right.  While  this 
is  within  acceptable  viewing  angles  for  critical  displays,  it  does  present  a  severe  parallax 
problem  when  another  person  tries  to  point  to  something  on  the  display.  In  the  interview, 
one  operator  commented:  "By  moving  the  graphic  screen  to  an  angle,  the  operator  would 
be  able  to  view  the  left  side  of  the  screen  much  more  easily."  Another  suggested:  "The 
keyboard  should  be  centered  so  there  is  an  equal  viewing  distance  to  the  screen." 

Working  Environment 

Physical  environment.  Operator  ratings  of  the  environment  are  shown  in  Table  12. 
However,  since  the  terminals  were  not  operated  in  the  shelters  for  this  test,  the  ratings  of 

TABLE  12 

Summary  of  Environment  Ratings 
(from  Human  Factors  Questionnaire) 


Number 

Ratings  of  adequate 

Mean 

Item 

responding 

or  better 

rating 

Number 

Percent 

Ventilation 

37 

35 

95 

4.08 

Vibration 

36 

34 

94 

4.11 

Noise 

37 

32 

86 

3.89 

Illumination 

37 

28 

76 

3.76 

Temperature 

37 

27 

73 

3.73 

28 


the  environment  are  of  limited  use.  Measurements  inside  the  three  shelters  housing  the 
system  equipment  indicated  the  operators  might  find  them  too  cold  and  too  noisy. 
Table  13  shows  the  temperature,  humidity  and  sound  levels  measured  on  three  days  w?hile 
the  system  was  operating.  All  of  the  relative  humidity  measures  were  acceptable,  being 
above  the  minimum  of  15  percent  required  by  the  standard  for  semipermanent  shelters. 
All  of  the  temperatures  in  the  processor  shelter  were  unacceptable,  being  below  the  65°F 
(18°C)  minimum  required.  These  temperatures  ranged  from  maximums  of  54.9°F  (12.7°C) 
to  64.2°F  (17.9°C).  The  communications  shelter  maximums  were  below  the  minimum  on 
two  of  the  three  days.  The  effect  of  the  low  temperatures  in  both  shelters  was 
aggravated  by  a  design  in  which  the  airconditioning  system  discharged  cold  air  directly  on 
the  operators.  This  type  of  design  is  specifically  prohibited  by  the  standard.  The  standard 
requires  a  maximum  noise  level  of  68  dB(A)  for  an  activity  where  "frequent  telephone  use 
or  frequent  direct  communicatons  at  distances  of  up  to  five  feet  is  required"  and  a 
maximum  of  75  dB(A)  where  occasional  communications  of  this  type  is  required.  The 
minimum  noise  level  measured  in  the  processor  van  shelter  during  the  three  days  was 
77  dB(A)  which  exceeds  both  conditions.  The  minimum  noise  level  in  the  communications 
shelter  was  68  dB(A)  which  exceeds  the  stricter  standard.  While  in  these  shelters,  the 
observer  noted  that  the  operators  had  a  great  deal  of  difficulty  talking  on  the  telephone. 
They  also  had  to  shout  at  each  other  to  be  understood  even  when  they  were  standing  very 
close  together.  The  environmental  measures  in  the  remote  display  station  shelter  were 
acceptable  in  every  case.  The  equipment  was  turned  on  but  it  was  not  being  used  for 
anything,  so  it  is  not  clear  that  the  same  desirable  conditions  would  be  obtained  with  six 
terminals  and  their  operators  in  action. 


TABLE  13 

Environmental  Measurements  of  JTFTB-A  Shelters 

Number  of  Day  Range 

Location  Measurements  1  2  3  Minimum  Maximum 


Mean  Dry  Bulb  Temperature  (Celcius) 


Central  Processor 

3 

17.0 

15.4 

16.4 

12.7 

17.9 

Communications 

2 

20.4 

14.2 

12.6 

12.2 

20.4 

Remote  Display 

2 

22.4 

20.6 

20.9 

20.2 

22.7 

Mean  Relative  Humidity  (percent) 


Central  Processor 

3 

48 

49 

53 

42 

56 

Communications 

2 

69 

51 

56 

48 

72 

Remote  Display 

2 

56 

63 

72 

56 

72 

r-y- 

v-1 

Mean  Noise  Levels  (dB(A)) 

*  *  *. 

Central  Processor 

3 

78 

78 

78 

77 

80 

Communications 

2 

72 

72 

70 

68 

74 

,  - 

**,."'* 

Remote  Display 

2 

54 

63 

63 

62 

64 

.** 

V- 

29 

£ 

* 

•  ,  •  %  ^  •  .  1  *  * 

*  »*  •  *  •  •  •  *  * 

*’  i 
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Physiological  problems.  A  summary  of  the  physical  complaints  acquired  during  the 
shifts  for  the  CPX  is  shown  In  Table  14.  The  most  common  complaints  at  the  end  of  shifts 
were  being  tired,  82.9  percent,  and  being  hungry,  44.7  percent.  Complaints  reported  by  at 
least  20  percent  of  the  operators  were  headaches,  eye  or  vision  problems,  neck  or  shoulder 
stiffness,  thirst,  and  back  problems.  Because  many  of  the  complaints  were  also  reported 
at  the  beginning  of  the  shift,  the  number  of  people  who  got  the  complaint  during  the  shift 
is  shown  in  the  table.  Getting  tired  or  drowsy  still  led  the  list  at  42.1  percent  followed  by 
hunger  at  35.5  percent.  Four  other  complaints  were  picked  up  by  more  than  10  percent  of 
the  operators  during  the  shift:  neck  or  shoulder  stiffness,  eye  or  vision  problems, 
headaches,  and  back  problems.  The  largest  number  of  complaints  were  picked  up  by 
operators  on  the  second  day  of  the  exercise  with  a  rate  of  2.29  per  person.  The  fewest 
was  1.71  on  the  first  day.  The  observers  obtained  the  complaint  information  by  interview 
and  it  appeared  that  many  of  the  operators  were  not  feeling  well  during  parts  of  the 
exercise. 


TABLE  14 

Percentage  of  Operators  Reporting  a  Specific  Complaint 
Who  Had  Not  Reported  it  at  the  Beginning  of  the  Shift 


Complaint 

Exercise 

1 

2 

3 

4 

Tired,  drowsy 

42.1 

38 

41 

55 

33 

Hungry 

35.5 

48 

29 

30 

33 

Neck  or  shoulders 

23.7 

24 

24 

20 

28 

Eye  or  vision 

22.4 

24 

29 

15 

22 

Headaches 

21.1 

14 

29 

15 

22 

Thirsty 

17.1 

5 

18 

15 

33 

Back 

17.1 

10 

24 

20 

17 

Cold/flu 

6.6 

0 

12 

5 

11 

Arms 

6.6 

10 

6 

5 

6 

Wrist  and  fingers 

6.6 

0 

18 

5 

6 

Stomach 

5.3 

0 

6 

0 

6 

Number  of  responses  76 

21 

17 

20 

18 

A  number  of  suggestions  for  changes  were  mentioned  in  the  earlier  discussions  but 
will  be  reviewed  again  briefly  here.  Among  the  more  frequently  mentioned  changes  were 
the  need  for  some  kind  of  signal  to  alert  the  operator  to  important  incoming  information, 
movement  of  the  joystick  to  make  it  easier  to  reach,  and  moving  the  ENTER  and  DONE 
keys  to  the  front  of  the  keyboard.  A  means  of  correcting  errors  more  simply  was  also  a 
commonly  expressed  need.  One  suggested  solution  was  to  add  a  "backup”  function  which 
would  allow  the  operator  to  return  to  a  previous  frame  without  having  to  CANCEL  and 
start  over.  The  need  for  clearer  error  messages  was  also  expressed. 
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The  addition  of  something  such  as  a  "hold  key"  was  suggested  as  a  means  to  allow 
the  operator  to  leave  a  function  and  come  back  to  it  at  the  point  in  the  dialogue  when 
he/she  left  the  function.  This  would  alleviate  the  need  to  CANCEL  and  start  over  if 
something  more  important  had  to  be  done. 

Software  change  suggestions  involved  simplifying  menus,  adding  default  selection, 
and  providing  a  means  to  skip  over  unwanted  frames.  An  interesting  software  addition  of 
a  "calculator  mode"  was  suggested  to  do  tasks  such  as  converting  map  coordinates.  One 
operator  brought  a  programmable  calculator  for  his  own  use  in  converting  UTM  to 
latitude-longitude  and  vice  versa. 

Numerous  suggestions  for  changes  in  the  construction  and  layout  of  the  terminal 
were  made.  However,  most  of  these  would  probably  have  not  been  made  if  the  terminal 
design  had  conformed  more  closely  with  the  standards  for  such  equipment. 

Several  operators  stated  a  need  for  making  some  of  the  functions  less  tedious  and 
time  consuming.  The  AGGREGATION,  PURGE,  and  TARGET  DESIGNATION  functions  in 
particular  were  cited.  Operators  disliked  using  these  functions  because  of  the  time 
required  if  several  entities  had  to  be  picked.  The  current  procedure  required  each  entity 
to  be  picked  separately.  A  procedure  which  would  permit  the  picking  of  all  the  entities 
before  further  processing  was  suggested  by  many  as  a  solution. 

Two  other  suggestions  of  a  minor  nature  were  made  by  a  number  of  operators.  One 
was  an  indicator  which  would  inform  the  operator  which  display  was  "attached"  to  the 
keyboard.  The  other  was  for  a  volume  control  for  the  headsets. 


Summary,  Observer  Comments,  and  Conclusions 

A  major  concern  in  evaluating  human  factors  characteristics  of  a  complex  system 
such  as  the  JTFTB-A  is  how  well  it  performs  the  job  it  was  designed  to  do.  A  system  can 
be  very  good  from  the  standpoint  of  how  comfortable  the  terminal  is  to  use  or  how 
"user-friendly"  the  dialogue  is  rated  and  yet  that  system  might  not  serve  user  needs  at  all. 
Conversely,  a  system  might  have  many  problems  on  both  of  these  aspects  and  still 
perform  its  job  adequately.  Obviously,  it  is  better  to  have  a  system  that  works  than  one 
that  is  easy  to  use  but  doesn't;  so  engineering  and  programming  problems  must  be  dealt 
with  first  in  the  development  process.  Human  factors  studies  of  the  type  described  here 
and,  even  more  desirably,  ones  that  make  more  direct,  specific  tests  of  various  features, 
can  then  point  the  way  to  make  future  versions  of  the  system  that  both  work  well  and  are 
easy  to  use. 

The  lack  of  a  concept  of  how  the  system  was  to  be  employed  or  any  specifications  of 
its  operational  capabilities  made  evaluation  of  all  objectives  in  the  field  test,  not  just  the 
human  factors  portion,  difficult  to  carry  out.  The  operators  in  this  test  were  not  well 
trained,  so  it  was  not  possible  to  get  direct  information  even  on  how  well  the  system  was 
capale  of  functioning  in  specific  cases.  Despite  these  limitations,  the  system  could  be 
compared  with  some  "baseline"  cases.  Most  operators  and  some  test  personnel  had  had 
experience  with  manual  intelligence  systems  in  the  field  against  which  they  could  judge 
the  CPX  performance.  There  was  also  a  notion  of  an  "ideal  ASAS"  based  on  experience 
with  other  automated  systems  that  the  operator  could  use  for  comparison.  Finally,  many 
of  the  human  factors  characteristics  could  be  compared  to  military  standards  or  good 
design  practices.  Such  implied  comparisons  cannot  be  used  to  say  that  a  system  is  good 
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but  do  provide  good  information  on  what  to  evaluate  more  closely  in  the  next  test  and 
what  design  changes  should  be  considered. 

Most  of  the  operators  thought  that  the  system  would  be  superior  to  the  manual 
methods  now  being  employed  by  intelligence  units.  They  liked  the  rapid  processing  of 
messages  and  the  ability  to  get  information  on  a  map  display  quickly  and  accurately. 
Their  difficulty  in  getting  it  to  work  in  the  CPX  and  the  time  it  took  to  complete  many 
functions  were  seen  as  negative  qualities.  They  were  concerned  about  their  minimal 
training  and  lack  of  knowledge  about  the  system.  The  opinions  of  the  system  dropped 
considerably  during  the  CPX  when  the  operators  were  mostly  unsuccessful  in  tracking  the 
enemy  forces. 

The  operators  were  trained  on  the  use  of  specific  functions,  so  these  could  be 
evaluated.  Most  of  this  data  were  collected  shortly  after  training,  before  the  operators 
had  had  much  experience  doing  anything  but  training  exercises.  As  the  observers  got  a 
better  understanding  of  the  system  by  watching  it  in  use  and  reviewing  the  technical 
materials,  it  became  clear  that  many  of  the  operators'  problems  were  the  result  of  not 
understanding  how  to  do  things  rather  than  any  deficiencies  in  the  system  design.  Thus, 
their  comments  must  be  viewed  from  a  perspective  of  learning  to  use  the  system 
functions  at  a  mechanical  level  rather  than  from  one  of  an  experienced  person  using  those 
functions  to  do  a  task. 

Almost  everyone  who  has  had  contact  with  the  operator  terminal  has  suggestions  on 
how  to  improve  its  design.  For  example,  the  first  group  of  operators  who  were  trained  at 
the  contractor's  facility  put  together  an  11  page  list  of  "system  enhancements."  Many  of 
these  same  "improvements"  were  brought  up  during  the  interviews  with  the  current  test 
participants.  It  is  not  possible  to  fully  evaluate  the  merits  of  any  of  the  suggested 
changes  with  the  limited  amount  of  testing  done  so  far.  Many  seem  to  fall  into  the 
category  of  an  alternative  approach  to  doing  the  same  task  rather  than  an  improvement  in 
the  performance  of  the  user-system  combination.  Most  of  the  operators  had  not  received 
enough  training  or  experience  to  know  how  to  carry  out  some  tasks  that  they  wanted  to 
do,  and  so  suggested  a  "function  key"  that  would  do  that  task.  While  such  functions  could 
facilitate  specific  tasks,  the  overall  effect  might  not  improve  system  performance. 

The  characteristics  of  the  alphanumeric  display  which  was  used  to  conduct  the 
dialogue  and  receive  messages  were  rated  favorably  by  most  operators.  One  exception 
was  the  signal  that  the  operator  had  a  message.  This  alert  signal,  incrementing  a  counter 
on  the  display,  was  missed  by  many  operators  when  they  were  involved  in  another  activity. 
The  characteristics  of  the  situation  display,  which  presented  the  location  of  units  on  a 
map  background,  also  were  rated  favorably  by  most  operators.  The  one  exception  for  this 
display  was  the  cursor,  which  the  operator  could  "lose"  if  it  was  moved  to  the  edge  of  the 
display. 

Individual  functions  were  rated  highly  by  most  operators  although  there  were  many 
suggestions  about  how  to  change  them.  The  numeric  key  cluster  was  considered  a  very 
useful  feature,  while  the  joystick  was  not  rated  as  adequate  by  many  because  of  the  large 
jumps  it  caused  the  curror  to  make  on  the  situation  display.  Some  operators  were  aware 
they  could  correct  this  latter  problem  by  executing  a  function  which  would  switch  to  a 
smooth  movement  mode.  The  operators  also  wanted  some  sort  of  indicator  on  the 
terminal  to  tell  them  which  display  was  attached  to  the  keyboard  at  that  time. 

Complex  functions  like  the  ones  used  to  build  queries  were  rated  highly  by  the 
operators,  but  again  they  had  many  minor  complaints  and  suggestions  for  improvement. 


Their  biggest  concern  was  the  time  it  took  to  go  through  the  functions  and  they  suggested 
many  shortcut  procedures  such  as  additional  functions  or  dialogue  frame  skipping 
procedures.  The  prompting  form  of  the  dialogue  made  it  easy  to  go  through  most 
functions  provided  the  operator  knew  which  function  was  needed  to  carry  out  a  task.  A 
minor  difficulty  was  that  some  seemingly  reasonable  entries  such  as  2400  hours  or  360 
degrees  would  not  be  accepted,  yuery  building  and  modification  functions  were  rated  as 
tedious  by  some.  A  suggested  improvement  was  a  listing  of  the  selections  and  defaults  at 
the  beginning  of  the  procedure.  The  aggregation  function  was  considered  too  difficult  to 
use  during  the  pressure  of  an  operation.  Some  means  to  pick  all  of  the  "children"  for  a 
"parent”  with  a  single  command  sequence  was  suggested  by  more  than  one  operator. 

The  design  of  the  terminal  was  compared  with  current  human  factors  standards.  It 
was  adequate  in  most  respects.  The  large  displays  and  the  color  coding  on  the  situation 
displays  made  information  readily  available  to  the  operator.  All  of  the  controls  needed  by 
the  operator  were  on  the  shelf  in  front  of  the  displays  and  a  secure  voice  communication 
system  let  the  operators  talk  with  one  another  and  with  other  elements  of  the  system. 

There  were  several  shortcomings  in  the  physical  configuration  of  the  terminal 
housing,  however.  It  was  not  designed  to  work  with  the  range  of  adult  male  sizes  required 
by  military  standards.  Also,  this  was  a  problem  for  some  of  the  operators  in  this  test  who 
were  short  females  and  had  difficulties  in  reaching  some  parts  of  the  keyboard.  The 
terminal  is  too  high  off  the  floor  for  short  operators  unless  an  easily  adjustable  chair  with 
a  heel  catch  is  provided.  The  space  for  legs  and  shoes  under  the  terminal  was  insufficient 
for  taller  operators.  The  small  size  of  the  characters  on  the  alphanumeric  display  and  the 
distance  the  operator  had  to  sit  in  front  of  the  terminal  made  that  screen  difficult  to  read 
for  some.  This  sitting  position  also  put  several  important  function  keys  and  the  joystick 
out  of  reach  of  shorter  operators  unless  they  leaned  forward.  When  this  was  done,  the 
problem  of  having  to  look  at  the  situation  display  at  an  angle  to  the  right  was  aggravated. 
This  angle  of  view  made  it  difficult  for  another  person  to  point  to  something  on  the 
situation  display  so  that  the  operator  would  know  what  was  being  indicated. 

The  control  arrangement  was  not  liked  by  some  operators.  Some  of  the  most 
frequently  used  were  not  only  located  on  the  top  row  of  keys  which  was  out  of  reach  for 
some  operators,  but  also  were  located  apart  from  the  numeric  cluster  with  which  they 
were  most  frequently  used.  Several  keys  had  functions  that  duplicated  other  procedures 
within  functions  or  even  other  keys.  Most  operators  who  suggested  changes  wanted  the 
duplicate  functions  removed  to  simplify  the  keyboard.  The  alphanumeric  display  was 
considered  adequate  except  for  a  problem  of  glare  on  the  faceplate.  This  same  problem 
was  reported  for  the  situation  display.  Both  screens  are  located  in  a  position  that  would 
make  a  short  operator  have  to  look  above  the  preferred  line-of-sight.  There  were  some 
complaints  about  the  color  coding  on  the  map  display  and  there  is  no  way  for  the  operator 
to  adjust  the  color  or  contrast  of  the  display. 

The  terminal  is  awkward  for  lefthanders  since  the  writing  surface  and  joystick  are 
to  the  right  of  the  keyboard.  The  writing  surface  is  slightly  smaller  than  the  standards 
require,  and  like  the  keyboard,  is  too  high  above  the  floor  for  the  chairs  provided.  The 
operators  were  concerned  that  there  is  no  volume  control  on  the  headsets.  Measurements 
of  the  temperatures  and  noise  levels  in  the  equipment  shelters  which  housed  the  central 
processor  and  communication  equipment  showed  these  were  both  too  cold  and  too  noisy 
according  to  standards.  Cramped  working  space  and  the  need  to  type  would  mitigate 
against  using  warm  clothing.  Ear  protection  might  be  difficult  because  there  is  a  constant 
need  to  use  the  telephone  in  these  areas. 


The  operators  were  questioned  about  any  physical  complaints  during  the  shifts  for 
the  CPX.  Several  common  symptoms  such  as  headaches,  neck  and  shoulder  stiffness,  and 
vision  or  eyestrain  were  reported.  Such  complaints  could  indicate  the  seating  posture  at 
the  terminal  and  the  long  sessions  at  the  video  display  are  fatiguing.  They  could  also 
indicate  the  operators  were  under  a  lot  of  stress  during  the  exercise. 

During  the  CPX,  it  was  the  observers'  impression  that  the  JTFTB-A  system  had  the 
capability  to  do  almost  everything  the  operators  wanted  it  to  do.  This  is  not  to  say  that  a 
particular  operator  who  wanted  to  do  a  specific  thing  either  was  able  to  do  it  or  even  had 
been  trained  to  do  it.  Rather,  it  means  that  when  an  operator  brought  up  a  need  for  a 
certain  kind  of  information,  either  another  operator,  a  member  of  the  test  staff,  or  one  of 
the  contractor  support  personnel  usually  were  able  to  find  a  way  to  accomplish  it.  It  is 
not  clear  whether  the  operators  could  be  trained  to  do  some  of  these  specific  tasks 
reliably  or  whether  these  tasks  could  be  done  in  an  efficient  manner. 

The  operators,  both  during  the  testing  of  single  functions  and  during  the  CPX,  had 
learned  to  operate  only  some  of  the  functions,  and  in  some  cases  they  did  not  know  how  to 
do  all  of  these  well.  In  fact,  they  still  were  in  the  process  of  discovering  how  to  use  the 
system  to  obtain  and  analyze  intelligence  information  during  the  CPX  itself.  A  warrant 
officer  serving  as  the  supervisor  told  an  observer  that  on  one  shift  he  "didn't  even  work 
the  CPX"  but  rather  spent  his  time  "punching  buttons"  to  discover  how  to  write  queries 
that  could  be  used  to  track  a  particular  kind  of  unit.  In  another  case,  an  operator  spent 
most  of  the  night  trying  to  work  out  a  method  to  deal  with  the  large  number  of  radar 
entity  data  records  presented  on  the  situation  display.  The  aggregation  function  took  too 
long  to  keep  up  with  the  incoming  information,  but  by  purging  all  of  the  entities  on  the 
screen  and  replacing  them  with  a  single  compound  element  symbol,  it  was  possible  to  stay 
updated.  This  also  required  a  use  of  past  queries  that  had  not  been  covered  in  the 
training. 

The  lack  of  good  documentation  aggravated  the  problem  of  using  the  system.  The 
manuals  were  arranged  in  a  tutorial  format  which  did  not  lend  itself  to  looking  up  specific 
information.  There  were  few  indexes  in  the  manuals  and  only  brief  tables  of  contents.  It 
took  almost  an  hour  of  the  combined  efforts  of  two  senior  operators,  a  test  observer,  and 
a  software  support  specialist  to  determine  how  to  make  a  relative  time  entry  in  a  past 
query  which  would  retrieve  only  those  entities  reported  in  the  past  six  hours.  Some  sort 
of  ready  reference  could  be  provided  at  each  terminal  to  cover  those  functions  that  the 
operators  do  not  use  frequently  enough  to  remember  well.  Alternatively,  this  information 
could  be  made  available  through  a  "help"  function  in  the  system. 

The  difficulties  operators  had  getting  the  system  to  give  them  what  they  wanted 
quickly  disenchanted  many.  They  would  give  up  trying  and  leave  the  terminal  for 
extended  coffee  breaks.  Since  problems  are  going  to  be  part  of  any  new  system,  it  is 
essential  to  keep  the  operators  from  giving  up.  Those  operators  who  were  able  to  get  help 
kept  trying  to  do  things  and  eventually  were  able  to  do  most  of  the  things  they  wanted. 
This  indicates  that  someone  who  is  knowledgeable  about  the  terminals  and  the  system 
always  should  be  accessible  when  testing.  The  test  team  personnel  were  able  to  perform 
some  of  this  function  once  they  became  proficient  with  the  terminals,  but  it  still  would  be 
desirable  to  have  a  software  specialist  in  the  terminal  area  whenever  testing  or  training  is 
in  progress. 

The  feedback  to  the  operators  from  the  system  was  one  of  its  weakest  links.  When 
an  operator  did  not  receive  the  information  expected,  it  was  virtually  impossible  to  be 
certain  what  had  happened.  Functions  like  queries  should  present  a  concise  restatement 
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of  what  was  selected  when  the  preparation  is  completed  or  when  it  is  brought  back  for 
modification.  A  hard  copy  of  this  summary  could  then  be  checked  by  supervisors  if 
problems  have  occurred.  The  error  messages  represent  a  primary  type  of  feedback  and 
many  were  not  easy  to  understand,  which  sometimes  led  to  operators  making  changes  in 
already  correct  information.  Error  messages  from  the  processor  were  often  missed 
because  the  alert  queue  increment  was  not  noticed.  An  attention  getting  signal  such  as 
flashing  characters  should  be  used  since  undetected  errors  are  bound  to  affect  operation 
capabilities  at  times.  Good  error  messages  should  give  all  of  the  problems  that  the 
processor  can  detect  and  should  refer  the  operator  to  the  appropriate  place  in  the 
documentation  for  additional  help.  The  repetition  of  this  much  information  can  be  tedious 
for  commonly  made  errors,  so  the  designers  might  consider  using  a  multilevel  error 
reporting  system  which  gives  only  abbreviated  error  messages  which  can  be  expanded  at 
the  operator's  request. 

Another  problem  for  the  operators  was  the  time  it  took  to  carry  out  many  common 
operations.  For  example,  aggregation,  purge,  or  target  designation  required  a  sequence  of 
key  strokes  for  each  entity  involved  in  addition  to  the  time  it  took  to  position  the  cursor 
over  an  entity.  Any  operation  that  required  regeneration  of  the  display  could  take  an 
additional  30  seconds  or  more  if  there  was  a  complicated  map  in  the  system.  This  delay 
was  a  nuisance  when  the  operator  wanted  only  a  single  piece  of  information  like  the 
cursor  coordinates.  More  important  is  that  the  rate  at  which  an  operator  performs 
commonly  used  functions  is  going  to  set  the  limit  on  how  much  information  an  operator 
can  handle  in  a  given  period.  This  in  turn  sets  the  limits  on  how  much  information  can  be 
processed  by  the  system  as  a  whole  with  a  given  set  of  terminal  assignments.  Alterna¬ 
tively,  reliable  information  on  how  fast  these  functions  can  be  carried  out  could  be  used 
to  predict  how  many  operator  terminals  will  be  required.  It  appears  that  considerable 
simplification  might  be  built  into  many  common  functions.  Global  picking  of  all  entities 
of  a  given  type  in  the  area  of  interest  for  target  designation  or  aggregation  would  be  one 
possibility.  Another  is  to  indicate  the  cursor  coordinates  on  the  alphanumeric  screen, 
which  would  eliminate  the  need  to  regenerate  the  situation  display. 

A  potentially  serious  problem  was  uncovered  during  the  single  function  testing  phase 
which  involved  the  possibility  that  more  than  one  operator  would  modify  the  same  entity 
data  record  during  the  same  time  period.  This  would  result  in  only  the  version  posted  last 
being  saved,  and  there  would  be  no  indication  to  the  other  operators  that  their  changes 
had  been  lost.  Discussion  of  this  problem  with  the  test  personnel  led  to  the  suggestion 
that  strict  operating  procedures  could  prevent  any  problems  with  duplicate  changes.  This 
does  not  seem  likely  because  the  dispersed  location  of  the  terminals  would  make  it 
difficult  to  monitor  all  of  the  operators.  In  addition,  the  time  pressure  on  the  operators 
to  get  the  job  done  would  make  it  virtually  impossible  to  specify  the  mutually  exclusive 
areas  of  responsibility  necessary  to  prevent  duplication.  It  is  likely  that  there  are  other 
possible  operations  where  the  work  of  one  operator  would  undo  that  of  another  without 
either  of  them  being  aware  of  it.  A  fairly  extensive  study  of  the  threats  to  the  integrity 
of  the  data  base  needs  to  be  undertaken.  This  would  allow  methods  of  protection  to  be 
devised  based  either  on  internal  system  controls  or  on  supervisory  methods. 

The  JTFTB-A  represents  an  excellent  vehicle  for  testing  the  human  factors 
elements  in  intelligence  decision  aiding  systems.  Future  tests  need  to  incorporate 
specific  experimental  tests  of  the  human  factors  issues  raised  in  this  first  field  test. 
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